[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:12:14 dist-test-slave-8hhm krb5kdc[7557](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:12:14 dist-test-slave-8hhm krb5kdc[7557](info): set up 2 sockets
Aug 11 00:12:14 dist-test-slave-8hhm krb5kdc[7557](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 00:12:15.577119 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.535s user 0.005s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:12:15 dist-test-slave-8hhm krb5kdc[7557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:12:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:12:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:12:15.781206 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:34309
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:45749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:34309
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:16.118657 7578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:16.119000 7578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:16.119170 7578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:16.126624 7578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:12:16.126758 7578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:16.126811 7578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:16.126904 7578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:12:16.126994 7578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:12:16.137024 7578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:45749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:34309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:34309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:16.138446 7578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:16.139957 7578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:16.152484 7594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:16.152549 7596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:16.152570 7603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:16.153342 7578 server_base.cc:1034] running on GCE node
I20260811 00:12:16.154202 7578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:16.155617 7578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:16.156829 7578 hybrid_clock.cc:648] HybridClock initialized: now 1786407136156776 us; error 37 us; skew 500 ppm
I20260811 00:12:16.157253 7578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:16 dist-test-slave-8hhm krb5kdc[7557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407136, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:16.160563 7578 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:12:16.162140 7578 webserver.cc:460] Webserver started at http://127.7.94.62:37041/ using document root <none> and password file <none>
I20260811 00:12:16.162899 7578 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:16.163057 7578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:16.163321 7578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:16.166515 7578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "8343399f831746bca71718b9d3ef2ea1"
format_stamp: "Formatted at 2026-08-11 00:12:16 on dist-test-slave-8hhm"
server_key: "905527cc61654420214c606bb525b86d"
server_key_iv: "254194d3d5ffef480bcd3cb554495db7"
server_key_version: "encryptionkey@0"
I20260811 00:12:16.167208 7578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "8343399f831746bca71718b9d3ef2ea1"
format_stamp: "Formatted at 2026-08-11 00:12:16 on dist-test-slave-8hhm"
server_key: "905527cc61654420214c606bb525b86d"
server_key_iv: "254194d3d5ffef480bcd3cb554495db7"
server_key_version: "encryptionkey@0"
I20260811 00:12:16.172389 7578 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:12:16.175809 7621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:16.177069 7578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:12:16.177279 7578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "8343399f831746bca71718b9d3ef2ea1"
format_stamp: "Formatted at 2026-08-11 00:12:16 on dist-test-slave-8hhm"
server_key: "905527cc61654420214c606bb525b86d"
server_key_iv: "254194d3d5ffef480bcd3cb554495db7"
server_key_version: "encryptionkey@0"
I20260811 00:12:16.177477 7578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:16.208153 7578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:16.212898 7578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:16.213294 7578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:16.233671 7578 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:34309
I20260811 00:12:16.233711 7708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:34309 every 8 connection(s)
I20260811 00:12:16.235080 7578 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:12:16.238917 7709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:16.241015 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 7578
I20260811 00:12:16.241361 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:12:16.243283 7544 external_mini_cluster.cc:1428] Setting key ba7f0de64b4f6e0a0b664a419f0f9247
I20260811 00:12:16.251592 7709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1: Bootstrap starting.
I20260811 00:12:16.255743 7709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:16.257112 7709 log.cc:824] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:16.260553 7709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1: No bootstrap required, opened a new log
I20260811 00:12:16.271059 7709 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8343399f831746bca71718b9d3ef2ea1" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34309 } }
I20260811 00:12:16.271472 7709 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:16.271665 7709 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8343399f831746bca71718b9d3ef2ea1, State: Initialized, Role: FOLLOWER
I20260811 00:12:16.272437 7709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8343399f831746bca71718b9d3ef2ea1" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34309 } }
I20260811 00:12:16.272754 7709 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:12:16.272923 7709 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:12:16.273142 7709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:16.276788 7709 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8343399f831746bca71718b9d3ef2ea1" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34309 } }
I20260811 00:12:16.277529 7709 leader_election.cc:302] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8343399f831746bca71718b9d3ef2ea1; no voters:
I20260811 00:12:16.278254 7709 leader_election.cc:288] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1 [CANDIDATE]: Term 1 election: Requested vote from peers
Aug 11 00:12:16 dist-test-slave-8hhm krb5kdc[7557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:16.278420 7716 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:16.279750 7716 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1 [term 1 LEADER]: Becoming Leader. State: Replica: 8343399f831746bca71718b9d3ef2ea1, State: Running, Role: LEADER
I20260811 00:12:16.280463 7716 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8343399f831746bca71718b9d3ef2ea1" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34309 } }
I20260811 00:12:16.280829 7709 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:12:16.290979 7720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8343399f831746bca71718b9d3ef2ea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8343399f831746bca71718b9d3ef2ea1" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34309 } } }
I20260811 00:12:16.291450 7720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1 [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:16.292006 7721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8343399f831746bca71718b9d3ef2ea1. Latest consensus state: current_term: 1 leader_uuid: "8343399f831746bca71718b9d3ef2ea1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8343399f831746bca71718b9d3ef2ea1" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34309 } } }
I20260811 00:12:16.292316 7721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1 [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:16.294857 7711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:16.247446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44562 (local address 127.7.94.62:34309)
0811 00:12:16.249159 (+ 1713us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:16.249175 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:16.249745 (+ 570us) server_negotiation.cc:407] Connection header received
0811 00:12:16.262248 (+ 12503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:16.262270 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:16.262687 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:16.263099 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:16.267216 (+ 4117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:16.268531 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:16.271166 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:16.272063 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:16.272219 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:16.280427 (+ 8208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:16.280450 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:16.280469 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:16.280533 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:16.285984 (+ 5451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:16.287626 (+ 1642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:16.287639 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:16.287651 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:16.287973 (+ 322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:16.288903 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:16.288915 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:16.288921 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:16.290072 (+ 1151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:16.290486 (+ 414us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:16.292206 (+ 1720us) server_negotiation.cc:299] Negotiation successful
0811 00:12:16.292834 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1470,"thread_start_us":1111,"threads_started":1}
I20260811 00:12:16.298525 7733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:12:16.304265 7733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:12:16.313504 7733 catalog_manager.cc:1347] Generated new cluster ID: 36ee5f417d824e89acc52b1398dcd7e8
I20260811 00:12:16.313673 7733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:12:16.332288 7733 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:12:16.333712 7733 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:12:16.347172 7733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8343399f831746bca71718b9d3ef2ea1: Generated new TSK 0
I20260811 00:12:16.348006 7733 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
I20260811 00:12:16.413316 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:34309
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:45749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:16.774261 7762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:16.774556 7762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:16.774662 7762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:16.782167 7762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:16.782280 7762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:16.782413 7762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:12:16.793735 7762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:45749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:34309
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:16.795181 7762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:16.796578 7762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:16.811906 7772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:16.812417 7770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:16.812419 7776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:16.812665 7762 server_base.cc:1034] running on GCE node
I20260811 00:12:16.813457 7762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:16.814380 7762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:16.815613 7762 hybrid_clock.cc:648] HybridClock initialized: now 1786407136815562 us; error 39 us; skew 500 ppm
I20260811 00:12:16.815937 7762 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:16 dist-test-slave-8hhm krb5kdc[7557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407136, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:16.819123 7762 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:12:16.820539 7762 webserver.cc:460] Webserver started at http://127.7.94.1:38069/ using document root <none> and password file <none>
I20260811 00:12:16.821300 7762 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:16.821450 7762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:16.821794 7762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:16.824697 7762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "955ee851e3874be9a8e79a5c93a54d96"
format_stamp: "Formatted at 2026-08-11 00:12:16 on dist-test-slave-8hhm"
server_key: "2fd43901957c472965f0283a0dea7fce"
server_key_iv: "201bc90e583f82facbe803c50e2cc357"
server_key_version: "encryptionkey@0"
I20260811 00:12:16.825537 7762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "955ee851e3874be9a8e79a5c93a54d96"
format_stamp: "Formatted at 2026-08-11 00:12:16 on dist-test-slave-8hhm"
server_key: "2fd43901957c472965f0283a0dea7fce"
server_key_iv: "201bc90e583f82facbe803c50e2cc357"
server_key_version: "encryptionkey@0"
I20260811 00:12:16.830856 7762 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 00:12:16.834372 7786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:16.835698 7762 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:12:16.835882 7762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "955ee851e3874be9a8e79a5c93a54d96"
format_stamp: "Formatted at 2026-08-11 00:12:16 on dist-test-slave-8hhm"
server_key: "2fd43901957c472965f0283a0dea7fce"
server_key_iv: "201bc90e583f82facbe803c50e2cc357"
server_key_version: "encryptionkey@0"
I20260811 00:12:16.836088 7762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:16.855422 7762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:16.859432 7762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:16.859725 7762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:16.860671 7762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:16.862115 7762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:16.862232 7762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:16.862382 7762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:16.862454 7762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:16.902968 7762 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:44443
I20260811 00:12:16.903015 7977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:44443 every 8 connection(s)
I20260811 00:12:16.904311 7762 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:12:16.905813 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 7762
I20260811 00:12:16.906059 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:12:16.906798 7544 external_mini_cluster.cc:1428] Setting key 05fe132bbf566d034fda021027c055e4
Aug 11 00:12:16 dist-test-slave-8hhm krb5kdc[7557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407136, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:16.924960 7985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:16.907442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:41495 (local address 127.7.94.62:34309)
0811 00:12:16.908011 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:16.908017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:16.909144 (+ 1127us) server_negotiation.cc:407] Connection header received
0811 00:12:16.910986 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:16.910991 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:16.911094 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:16.911219 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:16.912840 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:16.913652 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:16.915220 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:16.915644 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:16.915788 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:16.919328 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:16.919333 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:16.919337 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:16.919368 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:16.921997 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:16.922707 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:16.922715 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:16.922721 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:16.922846 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:16.923270 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:16.923276 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:16.923280 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:16.923537 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:16.923693 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:16.924371 (+ 678us) server_negotiation.cc:299] Negotiation successful
0811 00:12:16.924592 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":302,"threads_started":1}
WARNING: no policy specified for kudu/127.7.94.2@KRBTEST.COM; defaulting to no policy
I20260811 00:12:16.926026 7986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:16.908107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:34309 (local address 127.7.94.1:41495)
0811 00:12:16.908994 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:16.909034 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:16.910734 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:16.911494 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:16.911502 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:16.912061 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:16.912676 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:16.912694 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:16.913833 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:16.913836 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:16.915072 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:16.915084 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:16.915901 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:16.915908 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:16.916130 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:16.916807 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:16.916834 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:16.919102 (+ 2268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:16.922143 (+ 3041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:16.922152 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:16.922168 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:16.922578 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:16.922962 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:16.922967 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:16.922971 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:16.923159 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:16.923675 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:16.923683 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:16.924010 (+ 327us) client_negotiation.cc:769] Sending connection context
0811 00:12:16.924280 (+ 270us) client_negotiation.cc:240] Negotiation successful
0811 00:12:16.924820 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":376,"threads_started":1}
I20260811 00:12:16.927558 7978 heartbeater.cc:347] Connected to a master server at 127.7.94.62:34309
I20260811 00:12:16.927821 7978 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.2@KRBTEST.COM" created.
I20260811 00:12:16.928709 7978 heartbeater.cc:511] Master 127.7.94.62:34309 requested a full tablet report, sending...
I20260811 00:12:16.931061 7654 ts_manager.cc:194] Registered new tserver with Master: 955ee851e3874be9a8e79a5c93a54d96 (127.7.94.1:44443)
I20260811 00:12:16.932562 7654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:41495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
I20260811 00:12:16.952399 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:34309
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:45749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:17.292323 7992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:17.292605 7992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:17.292773 7992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:17.300190 7992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:17.300369 7992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:17.300581 7992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:12:17.310283 7992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:45749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:34309
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:17.311645 7992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:17.313069 7992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:17.326535 8004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:17.326819 8005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:17.326747 8007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:17.328219 7992 server_base.cc:1034] running on GCE node
I20260811 00:12:17.328846 7992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:17.329883 7992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:17.331112 7992 hybrid_clock.cc:648] HybridClock initialized: now 1786407137331054 us; error 64 us; skew 500 ppm
I20260811 00:12:17.331426 7992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:17 dist-test-slave-8hhm krb5kdc[7557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407137, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:17.334671 7992 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:12:17.336237 7992 webserver.cc:460] Webserver started at http://127.7.94.2:39965/ using document root <none> and password file <none>
I20260811 00:12:17.337013 7992 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:17.337168 7992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:17.337538 7992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:17.340723 7992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "066fbe653d074237b46bc738ace1dfd0"
format_stamp: "Formatted at 2026-08-11 00:12:17 on dist-test-slave-8hhm"
server_key: "9570486391ee0ae26b321bab63335b67"
server_key_iv: "e2c99de1a4ad0a9726b2896aad913770"
server_key_version: "encryptionkey@0"
I20260811 00:12:17.341588 7992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "066fbe653d074237b46bc738ace1dfd0"
format_stamp: "Formatted at 2026-08-11 00:12:17 on dist-test-slave-8hhm"
server_key: "9570486391ee0ae26b321bab63335b67"
server_key_iv: "e2c99de1a4ad0a9726b2896aad913770"
server_key_version: "encryptionkey@0"
I20260811 00:12:17.346879 7992 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:12:17.350894 8018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:17.352233 7992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 00:12:17.352401 7992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "066fbe653d074237b46bc738ace1dfd0"
format_stamp: "Formatted at 2026-08-11 00:12:17 on dist-test-slave-8hhm"
server_key: "9570486391ee0ae26b321bab63335b67"
server_key_iv: "e2c99de1a4ad0a9726b2896aad913770"
server_key_version: "encryptionkey@0"
I20260811 00:12:17.352612 7992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:17.375735 7992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:17.379870 7992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:17.380193 7992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:17.381664 7992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:17.383142 7992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:17.383251 7992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:17.383383 7992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:17.383460 7992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:17.419392 7992 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:36797
I20260811 00:12:17.419451 8208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:36797 every 8 connection(s)
I20260811 00:12:17.420748 7992 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:12:17.421991 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 7992
I20260811 00:12:17.422205 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:12:17.422784 7544 external_mini_cluster.cc:1428] Setting key bf5a6249bbc420c8411831814919714d
Aug 11 00:12:17 dist-test-slave-8hhm krb5kdc[7557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407137, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:17.444027 7985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:17.423825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:35281 (local address 127.7.94.62:34309)
0811 00:12:17.424054 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:17.424061 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:17.425450 (+ 1389us) server_negotiation.cc:407] Connection header received
0811 00:12:17.427678 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:17.427685 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:17.427773 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:17.427892 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:17.429494 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:17.430216 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:17.432397 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:17.433084 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:17.433283 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:17.437393 (+ 4110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:17.437398 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:17.437402 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:17.437435 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:17.440269 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:17.441422 (+ 1153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:17.441426 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:17.441428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:17.441616 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:17.442236 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:17.442240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:17.442242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:17.442506 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:17.442626 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:17.443599 (+ 973us) server_negotiation.cc:299] Negotiation successful
0811 00:12:17.443780 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
WARNING: no policy specified for kudu/127.7.94.3@KRBTEST.COM; defaulting to no policy
I20260811 00:12:17.445494 8215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:17.424446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:34309 (local address 127.7.94.2:35281)
0811 00:12:17.425310 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:17.425353 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:17.427376 (+ 2023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:17.428140 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:17.428155 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:17.428682 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:17.429318 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:17.429338 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:17.430366 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:17.430371 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:17.432235 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:17.432253 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:17.433359 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:17.433364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:17.433561 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:17.434281 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:17.434305 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:17.437112 (+ 2807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:17.440413 (+ 3301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:17.440425 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:17.440447 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:17.441297 (+ 850us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:17.441795 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:17.441800 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:17.441804 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:17.442117 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:17.442644 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:17.442656 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:17.443146 (+ 490us) client_negotiation.cc:769] Sending connection context
0811 00:12:17.443500 (+ 354us) client_negotiation.cc:240] Negotiation successful
0811 00:12:17.444116 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":366,"threads_started":1}
I20260811 00:12:17.447005 8210 heartbeater.cc:347] Connected to a master server at 127.7.94.62:34309
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.3@KRBTEST.COM" created.
I20260811 00:12:17.447362 8210 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:17.448097 8210 heartbeater.cc:511] Master 127.7.94.62:34309 requested a full tablet report, sending...
I20260811 00:12:17.449605 7654 ts_manager.cc:194] Registered new tserver with Master: 066fbe653d074237b46bc738ace1dfd0 (127.7.94.2:36797)
I20260811 00:12:17.450417 7654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:35281
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
I20260811 00:12:17.472090 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:34309
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:45749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:17.811291 8225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:17.811573 8225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:17.811669 8225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:17.818821 8225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:17.818924 8225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:17.819037 8225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:12:17.828449 8225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:45749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:34309
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:17.829808 8225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:17.831097 8225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:17.844576 8233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:17.844609 8234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:17.844519 8238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:17.845783 8225 server_base.cc:1034] running on GCE node
I20260811 00:12:17.846342 8225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:17.847237 8225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:17.848469 8225 hybrid_clock.cc:648] HybridClock initialized: now 1786407137848412 us; error 53 us; skew 500 ppm
I20260811 00:12:17.848825 8225 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:17 dist-test-slave-8hhm krb5kdc[7557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407137, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:17.852231 8225 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:12:17.853833 8225 webserver.cc:460] Webserver started at http://127.7.94.3:37341/ using document root <none> and password file <none>
I20260811 00:12:17.854617 8225 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:17.854759 8225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:17.855050 8225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:17.857985 8225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "805f7a0aea0c4b14ac9808164c75011b"
format_stamp: "Formatted at 2026-08-11 00:12:17 on dist-test-slave-8hhm"
server_key: "cdf80234c809f1db8a1bc4fc5c796060"
server_key_iv: "089fd776a436e0ef29f88c3ddcef1b3e"
server_key_version: "encryptionkey@0"
I20260811 00:12:17.858700 8225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "805f7a0aea0c4b14ac9808164c75011b"
format_stamp: "Formatted at 2026-08-11 00:12:17 on dist-test-slave-8hhm"
server_key: "cdf80234c809f1db8a1bc4fc5c796060"
server_key_iv: "089fd776a436e0ef29f88c3ddcef1b3e"
server_key_version: "encryptionkey@0"
I20260811 00:12:17.863808 8225 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:12:17.867053 8247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:17.868324 8225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:12:17.868453 8225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "805f7a0aea0c4b14ac9808164c75011b"
format_stamp: "Formatted at 2026-08-11 00:12:17 on dist-test-slave-8hhm"
server_key: "cdf80234c809f1db8a1bc4fc5c796060"
server_key_iv: "089fd776a436e0ef29f88c3ddcef1b3e"
server_key_version: "encryptionkey@0"
I20260811 00:12:17.868637 8225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:17.892810 8225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:17.896848 8225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:17.897095 8225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:17.898027 8225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:17.899417 8225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:17.899492 8225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:17.899632 8225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:17.899696 8225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:17.935464 8225 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:41899
I20260811 00:12:17.935491 7978 heartbeater.cc:503] Master 127.7.94.62:34309 was elected leader, sending a full tablet report...
I20260811 00:12:17.935487 8457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:41899 every 8 connection(s)
I20260811 00:12:17.936834 8225 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:12:17.940836 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 8225
I20260811 00:12:17.941041 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:12:17.941573 7544 external_mini_cluster.cc:1428] Setting key e7d2281ee223dbf1a031eed676534a4a
Aug 11 00:12:17 dist-test-slave-8hhm krb5kdc[7557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407137, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:17.956987 7985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:17.939632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:41157 (local address 127.7.94.62:34309)
0811 00:12:17.939798 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:17.939803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:17.940995 (+ 1192us) server_negotiation.cc:407] Connection header received
0811 00:12:17.943262 (+ 2267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:17.943267 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:17.943357 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:17.943464 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:17.945235 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:17.945976 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:17.947480 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:17.947865 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:17.947997 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:17.951390 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:17.951395 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:17.951398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:17.951427 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:17.953841 (+ 2414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:17.954514 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:17.954520 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:17.954524 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:17.954654 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:17.955065 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:17.955071 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:17.955074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:17.955342 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:17.955503 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:17.956329 (+ 826us) server_negotiation.cc:299] Negotiation successful
0811 00:12:17.956619 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 00:12:17.958001 8460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:17.940070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:34309 (local address 127.7.94.3:41157)
0811 00:12:17.940835 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:17.940878 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:17.943001 (+ 2123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:17.943710 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:17.943739 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:17.944350 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:17.945052 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:17.945070 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:17.946130 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:17.946134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:17.947326 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:17.947338 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:17.948089 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:17.948098 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:17.948339 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:17.948988 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:17.949009 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:17.951182 (+ 2173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:17.953975 (+ 2793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:17.953984 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:17.953998 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:17.954397 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:17.954765 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:17.954770 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:17.954774 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:17.954954 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:17.955486 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:17.955493 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:17.955886 (+ 393us) client_negotiation.cc:769] Sending connection context
0811 00:12:17.956209 (+ 323us) client_negotiation.cc:240] Negotiation successful
0811 00:12:17.956798 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":360,"threads_started":1}
I20260811 00:12:17.959280 8458 heartbeater.cc:347] Connected to a master server at 127.7.94.62:34309
I20260811 00:12:17.959568 8458 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:17.960232 8458 heartbeater.cc:511] Master 127.7.94.62:34309 requested a full tablet report, sending...
I20260811 00:12:17.961652 7654 ts_manager.cc:194] Registered new tserver with Master: 805f7a0aea0c4b14ac9808164c75011b (127.7.94.3:41899)
I20260811 00:12:17.962445 7654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:41157
I20260811 00:12:17.969175 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 00:12:17 dist-test-slave-8hhm krb5kdc[7557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407137, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 00:12:18 dist-test-slave-8hhm krb5kdc[7557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407137, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:12:18.019291 8470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:18.003590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46944 (local address 127.7.94.1:44443)
0811 00:12:18.004423 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:18.004430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:18.004459 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:12:18.004524 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:18.004530 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:18.004759 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:18.004890 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:18.006497 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:18.007500 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:18.008705 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:18.009350 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:18.009451 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:18.013490 (+ 4039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:18.013502 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:18.013516 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:18.013554 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:18.016898 (+ 3344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:18.017542 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:18.017548 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:18.017550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:18.017680 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:18.018000 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:18.018004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:18.018006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:18.018317 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:18.018536 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:18.018787 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 00:12:18.018990 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":493,"threads_started":1}
W20260811 00:12:18.020576 7828 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46944
Aug 11 00:12:18 dist-test-slave-8hhm krb5kdc[7557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 00:12:18 dist-test-slave-8hhm krb5kdc[7557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:12:18.068374 8470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:18.055169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46956 (local address 127.7.94.1:44443)
0811 00:12:18.055418 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:18.055425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:18.055751 (+ 326us) server_negotiation.cc:407] Connection header received
0811 00:12:18.055891 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:18.055895 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:18.055961 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:18.056045 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:18.057543 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:18.058206 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:18.059360 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:18.059796 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:18.059903 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:18.063746 (+ 3843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:18.063751 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:18.063755 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:18.063786 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:18.066293 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:18.066971 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:18.066975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:18.066977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:18.067105 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:18.067399 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:18.067402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:18.067404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:18.067626 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:18.067802 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:18.068002 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 00:12:18.068160 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 00:12:18.069913 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 7762
I20260811 00:12:18.082808 8470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:18.070885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46968 (local address 127.7.94.1:44443)
0811 00:12:18.071035 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:18.071038 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:18.071290 (+ 252us) server_negotiation.cc:407] Connection header received
0811 00:12:18.071382 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:18.071385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:18.071435 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:18.071505 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:18.072667 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:18.073584 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:18.075167 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:18.075553 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:18.075643 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:18.077855 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:18.077859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:18.077861 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:18.077886 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:18.080194 (+ 2308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:18.080845 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:18.080850 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:18.080853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:18.080981 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:18.081441 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:18.081446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:18.081449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:18.081751 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:18.081978 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:18.082290 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 00:12:18.082478 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 00:12:18.083487 7970 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:18.165422 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 7762
I20260811 00:12:18.192749 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 7992
Aug 11 00:12:18 dist-test-slave-8hhm krb5kdc[7557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.2@KRBTEST.COM
I20260811 00:12:18.207846 8480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:18.193758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41908 (local address 127.7.94.2:36797)
0811 00:12:18.194377 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:18.194384 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:18.194410 (+ 26us) server_negotiation.cc:407] Connection header received
0811 00:12:18.194475 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:18.194482 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:18.194694 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:18.194826 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:18.195968 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:18.196937 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:18.198153 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:18.198799 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:18.198896 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:18.202717 (+ 3821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:18.202727 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:18.202740 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:18.202795 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:18.205550 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:18.206135 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:18.206142 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:18.206144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:18.206265 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:18.206603 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:18.206606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:18.206608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:18.206928 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:18.207165 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:18.207359 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 00:12:18.207559 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":309,"threads_started":1}
I20260811 00:12:18.209110 8202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:18.290953 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 7992
I20260811 00:12:18.316244 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 8225
Aug 11 00:12:18 dist-test-slave-8hhm krb5kdc[7557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.3@KRBTEST.COM
I20260811 00:12:18.332521 8483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:18.317329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52752 (local address 127.7.94.3:41899)
0811 00:12:18.317936 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:18.317943 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:18.317972 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:12:18.318033 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:18.318038 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:18.318253 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:18.318419 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:18.319684 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:18.320715 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:18.322257 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:18.322817 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:18.322964 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:18.327081 (+ 4117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:18.327092 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:18.327106 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:18.327137 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:18.329830 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:18.330441 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:18.330448 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:18.330450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:18.330564 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:18.330944 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:18.330948 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:18.330950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:18.331297 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:18.331612 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:18.331928 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 00:12:18.332205 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":297,"threads_started":1}
I20260811 00:12:18.333899 8447 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:18.416249 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 8225
I20260811 00:12:18.443375 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 7578
I20260811 00:12:18.444664 7703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:18.554797 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 7578
2026-08-11T00:12:18Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4572 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:12:18 dist-test-slave-8hhm krb5kdc[8493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:12:18 dist-test-slave-8hhm krb5kdc[8493](info): set up 2 sockets
Aug 11 00:12:18 dist-test-slave-8hhm krb5kdc[8493](info): commencing operation
krb5kdc: starting...
W20260811 00:12:19.886247 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:12:20 dist-test-slave-8hhm krb5kdc[8493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:12:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:12:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:12:20.082614 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:42445
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:40611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:42445
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:20.406003 8519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:20.406347 8519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:20.406522 8519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:20.413347 8519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:12:20.413487 8519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:20.413571 8519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:20.413657 8519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:12:20.413744 8519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:12:20.423436 8519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:40611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:42445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:42445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:20.424798 8519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:20.426146 8519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:20.437816 8527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:20.437729 8534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:20.437785 8530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:20.438951 8519 server_base.cc:1034] running on GCE node
I20260811 00:12:20.439764 8519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:20.441108 8519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:20.442301 8519 hybrid_clock.cc:648] HybridClock initialized: now 1786407140442257 us; error 40 us; skew 500 ppm
I20260811 00:12:20.442641 8519 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:20 dist-test-slave-8hhm krb5kdc[8493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407140, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:20.451531 8519 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:12:20.453042 8519 webserver.cc:460] Webserver started at http://127.7.94.62:37047/ using document root <none> and password file <none>
I20260811 00:12:20.453780 8519 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:20.453948 8519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:20.454272 8519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:20.457360 8519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "629c7fe769e14eac96baf1621e7e1d8a"
format_stamp: "Formatted at 2026-08-11 00:12:20 on dist-test-slave-8hhm"
server_key: "4fbd0a5f6f6799e4b592b5a3a3e53c19"
server_key_iv: "e36367f806cc2fb9ee69033a99417f7d"
server_key_version: "encryptionkey@0"
I20260811 00:12:20.458130 8519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "629c7fe769e14eac96baf1621e7e1d8a"
format_stamp: "Formatted at 2026-08-11 00:12:20 on dist-test-slave-8hhm"
server_key: "4fbd0a5f6f6799e4b592b5a3a3e53c19"
server_key_iv: "e36367f806cc2fb9ee69033a99417f7d"
server_key_version: "encryptionkey@0"
I20260811 00:12:20.463362 8519 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:12:20.466755 8546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:20.468007 8519 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 00:12:20.468197 8519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "629c7fe769e14eac96baf1621e7e1d8a"
format_stamp: "Formatted at 2026-08-11 00:12:20 on dist-test-slave-8hhm"
server_key: "4fbd0a5f6f6799e4b592b5a3a3e53c19"
server_key_iv: "e36367f806cc2fb9ee69033a99417f7d"
server_key_version: "encryptionkey@0"
I20260811 00:12:20.468392 8519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:20.489280 8519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:20.493767 8519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:20.494086 8519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:20.512455 8519 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:42445
I20260811 00:12:20.512491 8631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:42445 every 8 connection(s)
I20260811 00:12:20.513888 8519 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:12:20.517580 8634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:20.520596 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 8519
I20260811 00:12:20.520876 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:12:20.521706 7544 external_mini_cluster.cc:1428] Setting key 65972075454db3ce9fb89f8989cf1633
I20260811 00:12:20.530189 8634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a: Bootstrap starting.
I20260811 00:12:20.533880 8634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:20.535211 8634 log.cc:824] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:12:20 dist-test-slave-8hhm krb5kdc[8493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:20.538043 8634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a: No bootstrap required, opened a new log
I20260811 00:12:20.545758 8638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:20.523751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44880 (local address 127.7.94.62:42445)
0811 00:12:20.524484 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:20.524495 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:20.524534 (+ 39us) server_negotiation.cc:407] Connection header received
0811 00:12:20.526221 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:20.526243 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:20.526633 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:20.526958 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:20.528640 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:20.529990 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:20.531675 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:20.532336 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:20.532457 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:20.537405 (+ 4948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:20.537413 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:20.537427 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:20.537459 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:20.541432 (+ 3973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:20.542064 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:20.542077 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:20.542086 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:20.542288 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:20.542613 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:20.542619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:20.542623 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:20.543287 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:20.543610 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:20.543997 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 00:12:20.544424 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":345,"threads_started":1}
I20260811 00:12:20.548261 8634 raft_consensus.cc:348] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "629c7fe769e14eac96baf1621e7e1d8a" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42445 } }
I20260811 00:12:20.548625 8634 raft_consensus.cc:374] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:20.548705 8634 raft_consensus.cc:729] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 629c7fe769e14eac96baf1621e7e1d8a, State: Initialized, Role: FOLLOWER
I20260811 00:12:20.549188 8634 consensus_queue.cc:260] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "629c7fe769e14eac96baf1621e7e1d8a" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42445 } }
I20260811 00:12:20.549486 8634 raft_consensus.cc:388] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:12:20.549585 8634 raft_consensus.cc:482] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:12:20.549711 8634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:20.551769 8634 raft_consensus.cc:504] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "629c7fe769e14eac96baf1621e7e1d8a" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42445 } }
I20260811 00:12:20.552193 8634 leader_election.cc:302] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 629c7fe769e14eac96baf1621e7e1d8a; no voters:
I20260811 00:12:20.552690 8634 leader_election.cc:288] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:12:20.552872 8642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:20.553628 8642 raft_consensus.cc:686] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a [term 1 LEADER]: Becoming Leader. State: Replica: 629c7fe769e14eac96baf1621e7e1d8a, State: Running, Role: LEADER
I20260811 00:12:20.554529 8642 consensus_queue.cc:237] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "629c7fe769e14eac96baf1621e7e1d8a" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42445 } }
I20260811 00:12:20.554924 8634 sys_catalog.cc:564] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:12:20.557535 8643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "629c7fe769e14eac96baf1621e7e1d8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "629c7fe769e14eac96baf1621e7e1d8a" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42445 } } }
I20260811 00:12:20.557871 8643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:20.558306 8644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 629c7fe769e14eac96baf1621e7e1d8a. Latest consensus state: current_term: 1 leader_uuid: "629c7fe769e14eac96baf1621e7e1d8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "629c7fe769e14eac96baf1621e7e1d8a" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42445 } } }
I20260811 00:12:20.558655 8644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:20.560071 8653 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:12:20.564841 8653 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:12:20.574818 8653 catalog_manager.cc:1347] Generated new cluster ID: c8a3af7f6c6d42868a748cc8ba8e288b
I20260811 00:12:20.574931 8653 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:12:20.614743 8653 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:12:20.616077 8653 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:12:20.629446 8653 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 629c7fe769e14eac96baf1621e7e1d8a: Generated new TSK 0
I20260811 00:12:20.630296 8653 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
I20260811 00:12:20.685189 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:42445
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:40611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:21.056766 8671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:21.057046 8671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:21.057310 8671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:21.064605 8671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:21.064739 8671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:21.064951 8671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:12:21.074941 8671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:40611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:42445
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:21.076224 8671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:21.077572 8671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:21.090550 8681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:21.090485 8685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:21.090552 8682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:21.091389 8671 server_base.cc:1034] running on GCE node
I20260811 00:12:21.092092 8671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:21.093030 8671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:21.094259 8671 hybrid_clock.cc:648] HybridClock initialized: now 1786407141094192 us; error 74 us; skew 500 ppm
I20260811 00:12:21.094584 8671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:21 dist-test-slave-8hhm krb5kdc[8493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407141, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:21.097671 8671 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:12:21.099086 8671 webserver.cc:460] Webserver started at http://127.7.94.1:43875/ using document root <none> and password file <none>
I20260811 00:12:21.099753 8671 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:21.099901 8671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:21.100207 8671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:21.103302 8671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "db570f7976e140249fdbb52f38843be3"
format_stamp: "Formatted at 2026-08-11 00:12:21 on dist-test-slave-8hhm"
server_key: "9650a927b99a91fa120f8cb58edd2d8c"
server_key_iv: "b3128e7299acbfc2c8545ac9c700b944"
server_key_version: "encryptionkey@0"
I20260811 00:12:21.104130 8671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "db570f7976e140249fdbb52f38843be3"
format_stamp: "Formatted at 2026-08-11 00:12:21 on dist-test-slave-8hhm"
server_key: "9650a927b99a91fa120f8cb58edd2d8c"
server_key_iv: "b3128e7299acbfc2c8545ac9c700b944"
server_key_version: "encryptionkey@0"
I20260811 00:12:21.109176 8671 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 00:12:21.112473 8697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:21.113832 8671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:12:21.114038 8671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "db570f7976e140249fdbb52f38843be3"
format_stamp: "Formatted at 2026-08-11 00:12:21 on dist-test-slave-8hhm"
server_key: "9650a927b99a91fa120f8cb58edd2d8c"
server_key_iv: "b3128e7299acbfc2c8545ac9c700b944"
server_key_version: "encryptionkey@0"
I20260811 00:12:21.114261 8671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:21.131070 8671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:21.135190 8671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:21.135499 8671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:21.136513 8671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:21.137935 8671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:21.138047 8671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:21.138198 8671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:21.138273 8671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:21.172550 8671 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:39393
I20260811 00:12:21.172605 8900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:39393 every 8 connection(s)
I20260811 00:12:21.174067 8671 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:12:21.177028 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 8671
I20260811 00:12:21.177269 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:12:21.178110 7544 external_mini_cluster.cc:1428] Setting key bc7a830d93b0bbd03825a69fa4f707a6
Aug 11 00:12:21 dist-test-slave-8hhm krb5kdc[8493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407141, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:21.194828 8903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:21.177393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:48239 (local address 127.7.94.62:42445)
0811 00:12:21.177894 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:21.177903 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:21.178472 (+ 569us) server_negotiation.cc:407] Connection header received
0811 00:12:21.180597 (+ 2125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:21.180602 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:21.180687 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:21.180821 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:21.182453 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:21.183174 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:21.184715 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:21.185138 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:21.185303 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:21.189048 (+ 3745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:21.189052 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:21.189055 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:21.189085 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:21.191648 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:21.192396 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:21.192402 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:21.192407 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:21.192539 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:21.193057 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:21.193063 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:21.193066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:21.193418 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:21.193568 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:21.194258 (+ 690us) server_negotiation.cc:299] Negotiation successful
0811 00:12:21.194478 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":251,"threads_started":1}
I20260811 00:12:21.195765 8909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:21.177421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:42445 (local address 127.7.94.1:48239)
0811 00:12:21.178293 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:21.178350 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:21.180308 (+ 1958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:21.181076 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:21.181084 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:21.181697 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:21.182272 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:21.182291 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:21.183336 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:21.183344 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:21.184558 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:21.184571 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:21.185392 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:21.185401 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:21.185623 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:21.186316 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:21.186358 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:21.188836 (+ 2478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:21.191827 (+ 2991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:21.191835 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:21.191850 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:21.192266 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:21.192655 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:21.192660 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:21.192664 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:21.192934 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:21.193545 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:21.193552 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:21.193871 (+ 319us) client_negotiation.cc:769] Sending connection context
0811 00:12:21.194164 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 00:12:21.194662 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":342,"threads_started":1}
WARNING: no policy specified for kudu/127.7.94.2@KRBTEST.COM; defaulting to no policy
I20260811 00:12:21.197369 8901 heartbeater.cc:347] Connected to a master server at 127.7.94.62:42445
I20260811 00:12:21.197664 8901 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:21.198449 8901 heartbeater.cc:511] Master 127.7.94.62:42445 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.2@KRBTEST.COM" created.
I20260811 00:12:21.200874 8580 ts_manager.cc:194] Registered new tserver with Master: db570f7976e140249fdbb52f38843be3 (127.7.94.1:39393)
I20260811 00:12:21.202338 8580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:48239
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
I20260811 00:12:21.224342 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:42445
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:40611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:21.556411 8912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:21.556695 8912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:21.556910 8912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:21.564097 8912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:21.564235 8912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:21.564437 8912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:12:21.574774 8912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:40611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:42445
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:21.576185 8912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:21.577636 8912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:21.591012 8929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:21.591090 8919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:21.591045 8926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:21.592412 8912 server_base.cc:1034] running on GCE node
I20260811 00:12:21.592998 8912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:21.593931 8912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:21.595155 8912 hybrid_clock.cc:648] HybridClock initialized: now 1786407141595110 us; error 32 us; skew 500 ppm
I20260811 00:12:21.595465 8912 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:21 dist-test-slave-8hhm krb5kdc[8493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407141, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:21.598508 8912 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:12:21.599884 8912 webserver.cc:460] Webserver started at http://127.7.94.2:41525/ using document root <none> and password file <none>
I20260811 00:12:21.600538 8912 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:21.600690 8912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:21.600993 8912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:21.603806 8912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "ac9c778c414a4238ac615ebd904f9f2a"
format_stamp: "Formatted at 2026-08-11 00:12:21 on dist-test-slave-8hhm"
server_key: "97b16818bce7770b88f8d17ab3b5daa5"
server_key_iv: "905266ae84c73672a2dd7f9e44ac17fb"
server_key_version: "encryptionkey@0"
I20260811 00:12:21.604576 8912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "ac9c778c414a4238ac615ebd904f9f2a"
format_stamp: "Formatted at 2026-08-11 00:12:21 on dist-test-slave-8hhm"
server_key: "97b16818bce7770b88f8d17ab3b5daa5"
server_key_iv: "905266ae84c73672a2dd7f9e44ac17fb"
server_key_version: "encryptionkey@0"
I20260811 00:12:21.609431 8912 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260811 00:12:21.612679 8939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:21.613932 8912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:12:21.614128 8912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "ac9c778c414a4238ac615ebd904f9f2a"
format_stamp: "Formatted at 2026-08-11 00:12:21 on dist-test-slave-8hhm"
server_key: "97b16818bce7770b88f8d17ab3b5daa5"
server_key_iv: "905266ae84c73672a2dd7f9e44ac17fb"
server_key_version: "encryptionkey@0"
I20260811 00:12:21.614316 8912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:21.638356 8912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:21.642474 8912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:21.642874 8912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:21.643810 8912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:21.645071 8912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:21.645195 8912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:21.645382 8912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:21.645454 8912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:21.679855 8912 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:40049
I20260811 00:12:21.679935 9113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:40049 every 8 connection(s)
I20260811 00:12:21.681074 8912 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:12:21.682403 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 8912
I20260811 00:12:21.682602 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:12:21.683133 7544 external_mini_cluster.cc:1428] Setting key bd9b423296cd5d21a2d2fb50999ff08f
Aug 11 00:12:21 dist-test-slave-8hhm krb5kdc[8493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407141, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:21.702409 8903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:21.683841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:56857 (local address 127.7.94.62:42445)
0811 00:12:21.684055 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:21.684060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:21.685425 (+ 1365us) server_negotiation.cc:407] Connection header received
0811 00:12:21.687327 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:21.687335 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:21.687424 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:21.687532 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:21.688975 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:21.690117 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:21.691952 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:21.692388 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:21.692480 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:21.696609 (+ 4129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:21.696614 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:21.696617 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:21.696651 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:21.699362 (+ 2711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:21.700111 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:21.700118 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:21.700125 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:21.700248 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:21.700659 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:21.700665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:21.700669 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:21.700920 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:21.701071 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:21.701810 (+ 739us) server_negotiation.cc:299] Negotiation successful
0811 00:12:21.702054 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
WARNING: no policy specified for kudu/127.7.94.3@KRBTEST.COM; defaulting to no policy
I20260811 00:12:21.703377 9121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:21.684343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:42445 (local address 127.7.94.2:56857)
0811 00:12:21.685197 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:21.685319 (+ 122us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:21.687043 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:21.687737 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:21.687744 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:21.688259 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:21.688792 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:21.688810 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:21.690272 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:21.690276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:21.691792 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:21.691808 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:21.692626 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:21.692636 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:21.692960 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:21.693726 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:21.693759 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:21.696372 (+ 2613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:21.699495 (+ 3123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:21.699504 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:21.699517 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:21.699989 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:21.700360 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:21.700365 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:21.700368 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:21.700546 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:21.701053 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:21.701060 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:21.701411 (+ 351us) client_negotiation.cc:769] Sending connection context
0811 00:12:21.701699 (+ 288us) client_negotiation.cc:240] Negotiation successful
0811 00:12:21.702227 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":331,"threads_started":1}
I20260811 00:12:21.704823 9119 heartbeater.cc:347] Connected to a master server at 127.7.94.62:42445
I20260811 00:12:21.705104 9119 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.3@KRBTEST.COM" created.
I20260811 00:12:21.705946 9119 heartbeater.cc:511] Master 127.7.94.62:42445 requested a full tablet report, sending...
I20260811 00:12:21.707573 8580 ts_manager.cc:194] Registered new tserver with Master: ac9c778c414a4238ac615ebd904f9f2a (127.7.94.2:40049)
I20260811 00:12:21.708802 8580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:56857
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
I20260811 00:12:21.729714 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:42445
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:40611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:22.054167 9125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:22.054438 9125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:22.054595 9125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:22.061913 9125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:22.062014 9125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:22.062124 9125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:12:22.071837 9125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:40611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:42445
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:22.073125 9125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:22.074456 9125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:22.087257 9131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:22.087463 9134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:22.087491 9132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:22.088402 9125 server_base.cc:1034] running on GCE node
I20260811 00:12:22.089020 9125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:22.089932 9125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:22.091122 9125 hybrid_clock.cc:648] HybridClock initialized: now 1786407142091069 us; error 47 us; skew 500 ppm
I20260811 00:12:22.091468 9125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:22 dist-test-slave-8hhm krb5kdc[8493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407142, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:22.094623 9125 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:12:22.096050 9125 webserver.cc:460] Webserver started at http://127.7.94.3:43713/ using document root <none> and password file <none>
I20260811 00:12:22.096680 9125 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:22.096822 9125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:22.097153 9125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:22.099933 9125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "3b04d5d893fa40edaab859918fa6b981"
format_stamp: "Formatted at 2026-08-11 00:12:22 on dist-test-slave-8hhm"
server_key: "08e0887ea558d81fa6653a9167a04ee5"
server_key_iv: "bb57c7cd21e8c56fd5bf936316c44d6c"
server_key_version: "encryptionkey@0"
I20260811 00:12:22.100705 9125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "3b04d5d893fa40edaab859918fa6b981"
format_stamp: "Formatted at 2026-08-11 00:12:22 on dist-test-slave-8hhm"
server_key: "08e0887ea558d81fa6653a9167a04ee5"
server_key_iv: "bb57c7cd21e8c56fd5bf936316c44d6c"
server_key_version: "encryptionkey@0"
I20260811 00:12:22.105540 9125 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 00:12:22.108867 9150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:22.110126 9125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:12:22.110308 9125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "3b04d5d893fa40edaab859918fa6b981"
format_stamp: "Formatted at 2026-08-11 00:12:22 on dist-test-slave-8hhm"
server_key: "08e0887ea558d81fa6653a9167a04ee5"
server_key_iv: "bb57c7cd21e8c56fd5bf936316c44d6c"
server_key_version: "encryptionkey@0"
I20260811 00:12:22.110496 9125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:22.159072 9125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:22.163084 9125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:22.163424 9125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:22.164528 9125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:22.165874 9125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:22.165972 9125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:22.166116 9125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:22.166185 9125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:22.200263 9125 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:46185
I20260811 00:12:22.200309 9351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:46185 every 8 connection(s)
I20260811 00:12:22.201653 9125 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:12:22.205895 8901 heartbeater.cc:503] Master 127.7.94.62:42445 was elected leader, sending a full tablet report...
I20260811 00:12:22.209004 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 9125
I20260811 00:12:22.209201 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:12:22.209757 7544 external_mini_cluster.cc:1428] Setting key 22caa2548f72f2358c4f10bb4d8a64cf
Aug 11 00:12:22 dist-test-slave-8hhm krb5kdc[8493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407142, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:22.222721 9355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:22.204233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:48823 (local address 127.7.94.62:42445)
0811 00:12:22.204670 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:22.204674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:22.205775 (+ 1101us) server_negotiation.cc:407] Connection header received
0811 00:12:22.207963 (+ 2188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:22.207967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:22.208023 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:22.208110 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:22.209987 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.211213 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.213107 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.213579 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.213681 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:22.217183 (+ 3502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:22.217188 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:22.217191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:22.217271 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:22.220152 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:22.220803 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:22.220807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:22.220809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:22.220918 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:22.221344 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:22.221347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:22.221349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:22.221592 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:22.221709 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:22.222306 (+ 597us) server_negotiation.cc:299] Negotiation successful
0811 00:12:22.222458 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":214,"threads_started":1}
I20260811 00:12:22.223858 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:22.204825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:42445 (local address 127.7.94.3:48823)
0811 00:12:22.205577 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:22.205612 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:22.207719 (+ 2107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:22.208315 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:22.208323 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:22.208985 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:22.209786 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:22.209813 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.211383 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.211390 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:22.212977 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:22.212989 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.213760 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.213765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:22.213951 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:22.214662 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:22.214697 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:22.216996 (+ 2299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:22.220273 (+ 3277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:22.220279 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:22.220291 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:22.220706 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:22.221008 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:22.221011 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:22.221014 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:22.221188 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:22.221697 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:22.221702 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:22.221997 (+ 295us) client_negotiation.cc:769] Sending connection context
0811 00:12:22.222210 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 00:12:22.222557 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":347,"threads_started":1}
I20260811 00:12:22.225310 9352 heartbeater.cc:347] Connected to a master server at 127.7.94.62:42445
I20260811 00:12:22.225575 9352 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:22.226249 9352 heartbeater.cc:511] Master 127.7.94.62:42445 requested a full tablet report, sending...
I20260811 00:12:22.227558 8580 ts_manager.cc:194] Registered new tserver with Master: 3b04d5d893fa40edaab859918fa6b981 (127.7.94.3:46185)
I20260811 00:12:22.228339 8580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:48823
I20260811 00:12:22.236757 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:12:22.255725 9355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:22.243073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52224 (local address 127.7.94.62:42445)
0811 00:12:22.243250 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:22.243257 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:22.243664 (+ 407us) server_negotiation.cc:407] Connection header received
0811 00:12:22.243871 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:22.243876 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:22.243952 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:22.244073 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:22.245624 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.246328 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.247549 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.247931 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.248062 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:22.250516 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:22.250520 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:22.250523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:22.250552 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:22.252969 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:22.253665 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:22.253673 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:22.253679 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:22.253802 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:22.254201 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:22.254206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:22.254209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:22.254514 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:22.254773 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:22.255211 (+ 438us) server_negotiation.cc:299] Negotiation successful
0811 00:12:22.255435 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 00:12:22.261762 8580 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:12:22.269062 8580 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:12:22.296319 9376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:22.281377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:39393 (local address 127.0.0.1:36952)
0811 00:12:22.283635 (+ 2258us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:22.283677 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:22.283834 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:22.286693 (+ 2859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:22.286701 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:22.286731 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:22.287196 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:22.287209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.290055 (+ 2846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.290064 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:22.293122 (+ 3058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:22.293145 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.295074 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.295084 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:22.295316 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:22.295386 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 00:12:22.295506 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 00:12:22.295773 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2026,"mutex_wait_us":1,"thread_start_us":1880,"threads_started":1}
I20260811 00:12:22.297787 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:22.281623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36952 (local address 127.7.94.1:39393)
0811 00:12:22.285863 (+ 4240us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:22.285876 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:22.285912 (+ 36us) server_negotiation.cc:407] Connection header received
0811 00:12:22.286034 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:22.286046 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:22.286357 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:22.287398 (+ 1041us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:22.287691 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.289849 (+ 2158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.293424 (+ 3575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.294859 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.296910 (+ 2051us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:22.297166 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:22.297279 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 00:12:22.297444 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3941,"spinlock_wait_cycles":26368,"thread_start_us":3905,"threads_started":1}
I20260811 00:12:22.301944 8798 tablet_service.cc:1467] Processing CreateTablet for tablet 3443626282334becb961ec454a6f6c9c (DEFAULT_TABLE table=test-table [id=6711943545c245d2a2ed146c365ef3d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:22.303596 8798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3443626282334becb961ec454a6f6c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:22.304497 9379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:22.281655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:46185 (local address 127.0.0.1:43568)
0811 00:12:22.284818 (+ 3163us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:22.284842 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:22.285054 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:22.295295 (+ 10241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:22.295299 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:22.295325 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:22.295795 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:22.295807 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.298645 (+ 2838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.298650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:22.300783 (+ 2133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:22.300799 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.303748 (+ 2949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.303755 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:22.303941 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:22.303973 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 00:12:22.304056 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 00:12:22.304176 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3006,"thread_start_us":2852,"threads_started":1}
I20260811 00:12:22.305706 9381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:22.282372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43568 (local address 127.7.94.3:46185)
0811 00:12:22.294074 (+ 11702us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:22.294086 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:22.294128 (+ 42us) server_negotiation.cc:407] Connection header received
0811 00:12:22.294263 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:22.294274 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:22.294556 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:22.296022 (+ 1466us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:22.296365 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.298472 (+ 2107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.302044 (+ 3572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.303479 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.304838 (+ 1359us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:22.305067 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:22.305148 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 00:12:22.305318 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11486,"thread_start_us":11295,"threads_started":1}
I20260811 00:12:22.307220 9376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:22.284943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:40049 (local address 127.0.0.1:59670)
0811 00:12:22.296601 (+ 11658us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:22.296619 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:22.296713 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:22.298107 (+ 1394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:22.298114 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:22.298148 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:22.298379 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:22.298393 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.301082 (+ 2689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.301091 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:22.303716 (+ 2625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:22.303737 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.306461 (+ 2724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.306467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:22.306651 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:22.306687 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 00:12:22.306797 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 00:12:22.306948 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11560,"thread_start_us":21281,"threads_started":1}
I20260811 00:12:22.307941 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:22.285268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59670 (local address 127.7.94.2:40049)
0811 00:12:22.297194 (+ 11926us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:22.297205 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:22.297286 (+ 81us) server_negotiation.cc:407] Connection header received
0811 00:12:22.297402 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:22.297413 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:22.297699 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:22.298584 (+ 885us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:22.298914 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.300864 (+ 1950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.304036 (+ 3172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.306276 (+ 2240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.306465 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:22.306681 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:22.306852 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 00:12:22.307015 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11717,"thread_start_us":11458,"threads_started":1}
I20260811 00:12:22.309891 9239 tablet_service.cc:1467] Processing CreateTablet for tablet 3443626282334becb961ec454a6f6c9c (DEFAULT_TABLE table=test-table [id=6711943545c245d2a2ed146c365ef3d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:22.311075 9239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3443626282334becb961ec454a6f6c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:22.311743 9021 tablet_service.cc:1467] Processing CreateTablet for tablet 3443626282334becb961ec454a6f6c9c (DEFAULT_TABLE table=test-table [id=6711943545c245d2a2ed146c365ef3d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:22.313103 9021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3443626282334becb961ec454a6f6c9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:22.318802 9389 tablet_bootstrap.cc:492] T 3443626282334becb961ec454a6f6c9c P db570f7976e140249fdbb52f38843be3: Bootstrap starting.
I20260811 00:12:22.322595 9389 tablet_bootstrap.cc:654] T 3443626282334becb961ec454a6f6c9c P db570f7976e140249fdbb52f38843be3: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:22.324095 9389 log.cc:824] T 3443626282334becb961ec454a6f6c9c P db570f7976e140249fdbb52f38843be3: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:22.324985 9390 tablet_bootstrap.cc:492] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981: Bootstrap starting.
I20260811 00:12:22.327198 9392 tablet_bootstrap.cc:492] T 3443626282334becb961ec454a6f6c9c P ac9c778c414a4238ac615ebd904f9f2a: Bootstrap starting.
I20260811 00:12:22.328795 9390 tablet_bootstrap.cc:654] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:22.329098 9389 tablet_bootstrap.cc:492] T 3443626282334becb961ec454a6f6c9c P db570f7976e140249fdbb52f38843be3: No bootstrap required, opened a new log
I20260811 00:12:22.329392 9389 ts_tablet_manager.cc:1397] T 3443626282334becb961ec454a6f6c9c P db570f7976e140249fdbb52f38843be3: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260811 00:12:22.329953 9390 log.cc:824] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:22.331043 9392 tablet_bootstrap.cc:654] T 3443626282334becb961ec454a6f6c9c P ac9c778c414a4238ac615ebd904f9f2a: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:22.332422 9390 tablet_bootstrap.cc:492] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981: No bootstrap required, opened a new log
I20260811 00:12:22.332470 9392 log.cc:824] T 3443626282334becb961ec454a6f6c9c P ac9c778c414a4238ac615ebd904f9f2a: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:22.332693 9390 ts_tablet_manager.cc:1397] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 00:12:22.335882 9392 tablet_bootstrap.cc:492] T 3443626282334becb961ec454a6f6c9c P ac9c778c414a4238ac615ebd904f9f2a: No bootstrap required, opened a new log
I20260811 00:12:22.336161 9392 ts_tablet_manager.cc:1397] T 3443626282334becb961ec454a6f6c9c P ac9c778c414a4238ac615ebd904f9f2a: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260811 00:12:22.338861 9389 raft_consensus.cc:348] T 3443626282334becb961ec454a6f6c9c P db570f7976e140249fdbb52f38843be3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db570f7976e140249fdbb52f38843be3" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 39393 } } peers { permanent_uuid: "3b04d5d893fa40edaab859918fa6b981" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 46185 } } peers { permanent_uuid: "ac9c778c414a4238ac615ebd904f9f2a" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40049 } }
I20260811 00:12:22.339278 9389 raft_consensus.cc:374] T 3443626282334becb961ec454a6f6c9c P db570f7976e140249fdbb52f38843be3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:22.339370 9389 raft_consensus.cc:729] T 3443626282334becb961ec454a6f6c9c P db570f7976e140249fdbb52f38843be3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db570f7976e140249fdbb52f38843be3, State: Initialized, Role: FOLLOWER
I20260811 00:12:22.339980 9389 consensus_queue.cc:260] T 3443626282334becb961ec454a6f6c9c P db570f7976e140249fdbb52f38843be3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db570f7976e140249fdbb52f38843be3" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 39393 } } peers { permanent_uuid: "3b04d5d893fa40edaab859918fa6b981" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 46185 } } peers { permanent_uuid: "ac9c778c414a4238ac615ebd904f9f2a" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40049 } }
I20260811 00:12:22.341773 9389 ts_tablet_manager.cc:1428] T 3443626282334becb961ec454a6f6c9c P db570f7976e140249fdbb52f38843be3: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 00:12:22.343461 9390 raft_consensus.cc:348] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db570f7976e140249fdbb52f38843be3" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 39393 } } peers { permanent_uuid: "3b04d5d893fa40edaab859918fa6b981" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 46185 } } peers { permanent_uuid: "ac9c778c414a4238ac615ebd904f9f2a" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40049 } }
I20260811 00:12:22.343881 9390 raft_consensus.cc:374] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:22.343983 9390 raft_consensus.cc:729] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b04d5d893fa40edaab859918fa6b981, State: Initialized, Role: FOLLOWER
I20260811 00:12:22.344578 9390 consensus_queue.cc:260] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db570f7976e140249fdbb52f38843be3" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 39393 } } peers { permanent_uuid: "3b04d5d893fa40edaab859918fa6b981" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 46185 } } peers { permanent_uuid: "ac9c778c414a4238ac615ebd904f9f2a" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40049 } }
I20260811 00:12:22.345685 9352 heartbeater.cc:503] Master 127.7.94.62:42445 was elected leader, sending a full tablet report...
I20260811 00:12:22.346326 9390 ts_tablet_manager.cc:1428] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260811 00:12:22.346861 9392 raft_consensus.cc:348] T 3443626282334becb961ec454a6f6c9c P ac9c778c414a4238ac615ebd904f9f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db570f7976e140249fdbb52f38843be3" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 39393 } } peers { permanent_uuid: "3b04d5d893fa40edaab859918fa6b981" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 46185 } } peers { permanent_uuid: "ac9c778c414a4238ac615ebd904f9f2a" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40049 } }
I20260811 00:12:22.347330 9392 raft_consensus.cc:374] T 3443626282334becb961ec454a6f6c9c P ac9c778c414a4238ac615ebd904f9f2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:22.347448 9392 raft_consensus.cc:729] T 3443626282334becb961ec454a6f6c9c P ac9c778c414a4238ac615ebd904f9f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac9c778c414a4238ac615ebd904f9f2a, State: Initialized, Role: FOLLOWER
I20260811 00:12:22.348090 9392 consensus_queue.cc:260] T 3443626282334becb961ec454a6f6c9c P ac9c778c414a4238ac615ebd904f9f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db570f7976e140249fdbb52f38843be3" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 39393 } } peers { permanent_uuid: "3b04d5d893fa40edaab859918fa6b981" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 46185 } } peers { permanent_uuid: "ac9c778c414a4238ac615ebd904f9f2a" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40049 } }
I20260811 00:12:22.349179 9119 heartbeater.cc:503] Master 127.7.94.62:42445 was elected leader, sending a full tablet report...
I20260811 00:12:22.349754 9392 ts_tablet_manager.cc:1428] T 3443626282334becb961ec454a6f6c9c P ac9c778c414a4238ac615ebd904f9f2a: Time spent starting tablet: real 0.013s user 0.008s sys 0.007s
W20260811 00:12:22.426752 8902 tablet.cc:2369] T 3443626282334becb961ec454a6f6c9c P db570f7976e140249fdbb52f38843be3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:12:22.433385 9120 tablet.cc:2369] T 3443626282334becb961ec454a6f6c9c P ac9c778c414a4238ac615ebd904f9f2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:12:22.453330 9354 tablet.cc:2369] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:12:22.558485 9398 raft_consensus.cc:482] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:12:22.558817 9398 raft_consensus.cc:504] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db570f7976e140249fdbb52f38843be3" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 39393 } } peers { permanent_uuid: "3b04d5d893fa40edaab859918fa6b981" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 46185 } } peers { permanent_uuid: "ac9c778c414a4238ac615ebd904f9f2a" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40049 } }
I20260811 00:12:22.560565 9398 leader_election.cc:288] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db570f7976e140249fdbb52f38843be3 (127.7.94.1:39393), ac9c778c414a4238ac615ebd904f9f2a (127.7.94.2:40049)
I20260811 00:12:22.566495 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:22.561099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:39393 (local address 127.7.94.3:35575)
0811 00:12:22.561350 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:22.561370 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:22.561519 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:22.561869 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:22.561873 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:22.561906 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:22.562109 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:22.562118 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.563128 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.563137 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:22.564611 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:22.564624 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.565773 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.565779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:22.565974 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:22.565998 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 00:12:22.566114 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 00:12:22.566240 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 00:12:22.566495 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:22.561087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:35575 (local address 127.7.94.1:39393)
0811 00:12:22.561351 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:22.561358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:22.561448 (+ 90us) server_negotiation.cc:407] Connection header received
0811 00:12:22.561687 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:22.561691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:22.561751 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:22.561851 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:22.562252 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.562964 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.564776 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.565618 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.565762 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:22.565861 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:22.566135 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 00:12:22.566257 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 00:12:22.567360 9400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:22.561099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:40049 (local address 127.7.94.3:40245)
0811 00:12:22.562139 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:22.562159 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:22.562378 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:22.562705 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:22.562709 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:22.562731 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:22.563031 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:22.563039 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.564137 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.564142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:22.565603 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:22.565615 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.566672 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.566678 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:22.566845 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:22.566879 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 00:12:22.566960 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 00:12:22.567095 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":947,"mutex_wait_us":253,"thread_start_us":456,"threads_started":1}
I20260811 00:12:22.567425 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:22.561075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:40245 (local address 127.7.94.2:40049)
0811 00:12:22.561318 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:22.561325 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:22.562323 (+ 998us) server_negotiation.cc:407] Connection header received
0811 00:12:22.562523 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:22.562526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:22.562583 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:22.562692 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:22.563237 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.563966 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.565751 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.566468 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.566617 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:22.566713 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:22.567015 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 00:12:22.567168 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 00:12:22.567631 8826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3443626282334becb961ec454a6f6c9c" candidate_uuid: "3b04d5d893fa40edaab859918fa6b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db570f7976e140249fdbb52f38843be3" is_pre_election: true
I20260811 00:12:22.568216 8826 raft_consensus.cc:2454] T 3443626282334becb961ec454a6f6c9c P db570f7976e140249fdbb52f38843be3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b04d5d893fa40edaab859918fa6b981 in term 0.
I20260811 00:12:22.568466 9052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3443626282334becb961ec454a6f6c9c" candidate_uuid: "3b04d5d893fa40edaab859918fa6b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac9c778c414a4238ac615ebd904f9f2a" is_pre_election: true
I20260811 00:12:22.568991 9052 raft_consensus.cc:2454] T 3443626282334becb961ec454a6f6c9c P ac9c778c414a4238ac615ebd904f9f2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b04d5d893fa40edaab859918fa6b981 in term 0.
I20260811 00:12:22.568961 9152 leader_election.cc:302] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b04d5d893fa40edaab859918fa6b981, db570f7976e140249fdbb52f38843be3; no voters:
I20260811 00:12:22.569401 9398 raft_consensus.cc:2781] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:12:22.569557 9398 raft_consensus.cc:482] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:12:22.569671 9398 raft_consensus.cc:3037] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:22.572006 9398 raft_consensus.cc:504] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db570f7976e140249fdbb52f38843be3" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 39393 } } peers { permanent_uuid: "3b04d5d893fa40edaab859918fa6b981" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 46185 } } peers { permanent_uuid: "ac9c778c414a4238ac615ebd904f9f2a" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40049 } }
I20260811 00:12:22.572700 9398 leader_election.cc:288] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981 [CANDIDATE]: Term 1 election: Requested vote from peers db570f7976e140249fdbb52f38843be3 (127.7.94.1:39393), ac9c778c414a4238ac615ebd904f9f2a (127.7.94.2:40049)
I20260811 00:12:22.573069 8826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3443626282334becb961ec454a6f6c9c" candidate_uuid: "3b04d5d893fa40edaab859918fa6b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db570f7976e140249fdbb52f38843be3"
I20260811 00:12:22.573127 9052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3443626282334becb961ec454a6f6c9c" candidate_uuid: "3b04d5d893fa40edaab859918fa6b981" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac9c778c414a4238ac615ebd904f9f2a"
I20260811 00:12:22.573287 8826 raft_consensus.cc:3037] T 3443626282334becb961ec454a6f6c9c P db570f7976e140249fdbb52f38843be3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:22.573401 9052 raft_consensus.cc:3037] T 3443626282334becb961ec454a6f6c9c P ac9c778c414a4238ac615ebd904f9f2a [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:22.575546 8826 raft_consensus.cc:2454] T 3443626282334becb961ec454a6f6c9c P db570f7976e140249fdbb52f38843be3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b04d5d893fa40edaab859918fa6b981 in term 1.
I20260811 00:12:22.575695 9052 raft_consensus.cc:2454] T 3443626282334becb961ec454a6f6c9c P ac9c778c414a4238ac615ebd904f9f2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b04d5d893fa40edaab859918fa6b981 in term 1.
I20260811 00:12:22.575909 9152 leader_election.cc:302] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b04d5d893fa40edaab859918fa6b981, db570f7976e140249fdbb52f38843be3; no voters:
I20260811 00:12:22.576263 9398 raft_consensus.cc:2781] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:22.576975 9398 raft_consensus.cc:686] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981 [term 1 LEADER]: Becoming Leader. State: Replica: 3b04d5d893fa40edaab859918fa6b981, State: Running, Role: LEADER
I20260811 00:12:22.577517 9398 consensus_queue.cc:237] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db570f7976e140249fdbb52f38843be3" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 39393 } } peers { permanent_uuid: "3b04d5d893fa40edaab859918fa6b981" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 46185 } } peers { permanent_uuid: "ac9c778c414a4238ac615ebd904f9f2a" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40049 } }
I20260811 00:12:22.581872 8579 catalog_manager.cc:5495] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b04d5d893fa40edaab859918fa6b981 (127.7.94.3). New cstate: current_term: 1 leader_uuid: "3b04d5d893fa40edaab859918fa6b981" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db570f7976e140249fdbb52f38843be3" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 39393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b04d5d893fa40edaab859918fa6b981" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 46185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac9c778c414a4238ac615ebd904f9f2a" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40049 } health_report { overall_health: UNKNOWN } } }
Aug 11 00:12:22 dist-test-slave-8hhm krb5kdc[8493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:12:22.665419 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:22.650200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36962 (local address 127.7.94.1:39393)
0811 00:12:22.650400 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:22.650416 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:22.650888 (+ 472us) server_negotiation.cc:407] Connection header received
0811 00:12:22.651102 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:22.651106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:22.651196 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:22.651304 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:22.652898 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.653733 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.654990 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.655475 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.655622 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:22.659833 (+ 4211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:22.659850 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:22.659867 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:22.659898 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:22.662580 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:22.663223 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:22.663231 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:22.663233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:22.663388 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:22.663782 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:22.663787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:22.663791 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:22.664171 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:22.664488 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:22.664786 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 00:12:22.665006 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 00:12:22 dist-test-slave-8hhm krb5kdc[8493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407142, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 00:12:22 dist-test-slave-8hhm krb5kdc[8493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407142, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:12:22.718041 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:22.703717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36964 (local address 127.7.94.1:39393)
0811 00:12:22.703970 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:22.703976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:22.704239 (+ 263us) server_negotiation.cc:407] Connection header received
0811 00:12:22.704452 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:22.704465 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:22.704532 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:22.704649 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:22.706062 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.706785 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.708021 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.708462 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.708609 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:22.712345 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:22.712350 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:22.712353 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:22.712384 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:22.715069 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:22.715998 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:22.716003 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:22.716005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:22.716146 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:22.716539 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:22.716545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:22.716548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:22.716833 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:22.717123 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:22.717396 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 00:12:22.717607 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
Aug 11 00:12:22 dist-test-slave-8hhm krb5kdc[8493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 00:12:22 dist-test-slave-8hhm krb5kdc[8493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:12:22.770171 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:22.755855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36974 (local address 127.7.94.1:39393)
0811 00:12:22.756128 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:22.756138 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:22.756660 (+ 522us) server_negotiation.cc:407] Connection header received
0811 00:12:22.756885 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:22.756890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:22.756956 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:22.757070 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:22.758362 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.759032 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.760242 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.760671 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.760805 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:22.764707 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:22.764712 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:22.764716 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:22.764750 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:22.767490 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:22.768155 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:22.768161 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:22.768170 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:22.768286 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:22.768666 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:22.768671 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:22.768674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:22.768983 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:22.769282 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:22.769576 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 00:12:22.769817 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260811 00:12:22.772365 8751 tablet_service.cc:2908] Rejecting scan request for tablet 3443626282334becb961ec454a6f6c9c: Uninitialized: clean time has not yet been initialized
I20260811 00:12:22.775791 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 8671
I20260811 00:12:22.788071 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:22.776692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36982 (local address 127.7.94.1:39393)
0811 00:12:22.776911 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:22.776917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:22.777183 (+ 266us) server_negotiation.cc:407] Connection header received
0811 00:12:22.777376 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:22.777380 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:22.777453 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:22.777529 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:22.778748 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.779401 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.780543 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.780950 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.781052 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:22.783238 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:22.783243 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:22.783246 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:22.783281 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:22.785757 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:22.786337 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:22.786340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:22.786342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:22.786448 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:22.786752 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:22.786755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:22.786757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:22.786975 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:22.787299 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:22.787549 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 00:12:22.787764 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 00:12:22.788683 8893 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:22.880204 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 8671
I20260811 00:12:22.912871 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 8912
Aug 11 00:12:22 dist-test-slave-8hhm krb5kdc[8493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.2@KRBTEST.COM
I20260811 00:12:22.927702 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:22.913803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59674 (local address 127.7.94.2:40049)
0811 00:12:22.914049 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:22.914056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:22.914081 (+ 25us) server_negotiation.cc:407] Connection header received
0811 00:12:22.914156 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:22.914160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:22.914233 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:22.914317 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:22.915528 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.916244 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.917566 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:22.917928 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:22.918102 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:22.922118 (+ 4016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:22.922132 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:22.922150 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:22.922182 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:22.924805 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:22.925628 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:22.925636 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:22.925638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:22.925768 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:22.926165 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:22.926169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:22.926171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:22.926519 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:22.926823 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:22.927140 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 00:12:22.927367 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 00:12:22.928330 9107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:23.015005 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 8912
I20260811 00:12:23.026605 9400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:23.026040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:39393 (local address 127.7.94.3:51229)
0811 00:12:23.026262 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:23.026410 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:39393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 00:12:23.026933 9152 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.94.1:39393: connect: Connection refused (error 111)
I20260811 00:12:23.028525 9400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:23.028034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:39393 (local address 127.7.94.3:60871)
0811 00:12:23.028237 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:23.028337 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:39393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 00:12:23.028746 9152 consensus_peers.cc:487] T 3443626282334becb961ec454a6f6c9c P 3b04d5d893fa40edaab859918fa6b981 -> Peer db570f7976e140249fdbb52f38843be3 (127.7.94.1:39393): Couldn't send request to peer db570f7976e140249fdbb52f38843be3. Status: Network error: Client connection negotiation failed: client connection to 127.7.94.1:39393: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 00:12:23.042209 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 9125
Aug 11 00:12:23 dist-test-slave-8hhm krb5kdc[8493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.3@KRBTEST.COM
I20260811 00:12:23.056859 9432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:23.042946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43570 (local address 127.7.94.3:46185)
0811 00:12:23.043368 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:23.043373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:23.043392 (+ 19us) server_negotiation.cc:407] Connection header received
0811 00:12:23.043445 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:23.043449 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:23.043510 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:23.043599 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:23.044775 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:23.045548 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:23.046711 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:23.047109 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:23.047196 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:23.051291 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:23.051304 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:23.051321 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:23.051351 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:23.054280 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:23.054963 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:23.054970 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:23.054972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:23.055152 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:23.055473 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:23.055476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:23.055478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:23.055814 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:23.056077 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:23.056299 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 00:12:23.056464 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":226,"threads_started":1}
I20260811 00:12:23.057583 9342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:23.145705 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 9125
I20260811 00:12:23.174022 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 8519
I20260811 00:12:23.175226 8626 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:23.283548 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 8519
2026-08-11T00:12:23Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4729 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:12:23 dist-test-slave-8hhm krb5kdc[9440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:12:23 dist-test-slave-8hhm krb5kdc[9440](info): set up 2 sockets
Aug 11 00:12:23 dist-test-slave-8hhm krb5kdc[9440](info): commencing operation
krb5kdc: starting...
W20260811 00:12:24.615411 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.002s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:12:24 dist-test-slave-8hhm krb5kdc[9440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:12:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:12:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:12:24.814909 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:45393
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:43173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:45393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:25.135390 9461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:25.135748 9461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:25.135912 9461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:25.142800 9461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:12:25.142935 9461 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 00:12:25.143026 9461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:25.143136 9461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:25.143211 9461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:12:25.143299 9461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:12:25.152667 9461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:43173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:45393
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:45393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:25.153947 9461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:25.155275 9461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:25.166818 9473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:25.166929 9470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:25.166906 9468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:25.167892 9461 server_base.cc:1034] running on GCE node
I20260811 00:12:25.168617 9461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:25.169942 9461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:25.171181 9461 hybrid_clock.cc:648] HybridClock initialized: now 1786407145171116 us; error 36 us; skew 500 ppm
I20260811 00:12:25.171489 9461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:25 dist-test-slave-8hhm krb5kdc[9440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407145, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:25.174589 9461 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:12:25.176021 9461 webserver.cc:460] Webserver started at http://127.7.94.62:33799/ using document root <none> and password file <none>
I20260811 00:12:25.176692 9461 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:25.176816 9461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:25.177131 9461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:25.180182 9461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "8a6450d6ee5e4752a771d83b158a97a9"
format_stamp: "Formatted at 2026-08-11 00:12:25 on dist-test-slave-8hhm"
server_key: "d040da5a40ab78d7c527fbf0138e5a02"
server_key_iv: "9b86f7a2baef6887a42405d6ff19dc61"
server_key_version: "encryptionkey@0"
I20260811 00:12:25.180979 9461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "8a6450d6ee5e4752a771d83b158a97a9"
format_stamp: "Formatted at 2026-08-11 00:12:25 on dist-test-slave-8hhm"
server_key: "d040da5a40ab78d7c527fbf0138e5a02"
server_key_iv: "9b86f7a2baef6887a42405d6ff19dc61"
server_key_version: "encryptionkey@0"
I20260811 00:12:25.186025 9461 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 00:12:25.189293 9485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:25.190513 9461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:12:25.190693 9461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "8a6450d6ee5e4752a771d83b158a97a9"
format_stamp: "Formatted at 2026-08-11 00:12:25 on dist-test-slave-8hhm"
server_key: "d040da5a40ab78d7c527fbf0138e5a02"
server_key_iv: "9b86f7a2baef6887a42405d6ff19dc61"
server_key_version: "encryptionkey@0"
I20260811 00:12:25.190922 9461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:25.210451 9461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:25.214290 9461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:25.214572 9461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:25.236116 9461 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:45393
I20260811 00:12:25.236136 9582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:45393 every 8 connection(s)
I20260811 00:12:25.237512 9461 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:12:25.241060 9584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:25.243072 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 9461
I20260811 00:12:25.243296 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:12:25.243908 7544 external_mini_cluster.cc:1428] Setting key fa6af0706a8152fdef0dd1da39a47028
I20260811 00:12:25.254005 9584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9: Bootstrap starting.
I20260811 00:12:25.257701 9584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:25.258740 9584 log.cc:824] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:12:25 dist-test-slave-8hhm krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:25.261783 9584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9: No bootstrap required, opened a new log
I20260811 00:12:25.268071 9587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:25.246046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59834 (local address 127.7.94.62:45393)
0811 00:12:25.246900 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:25.246917 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:25.246970 (+ 53us) server_negotiation.cc:407] Connection header received
0811 00:12:25.249398 (+ 2428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:25.249428 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:25.249914 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:25.250366 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:25.251596 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:25.253574 (+ 1978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:25.254832 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:25.255953 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:25.256134 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:25.260427 (+ 4293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:25.260435 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:25.260449 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:25.260482 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:25.263700 (+ 3218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:25.264521 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:25.264527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:25.264535 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:25.264698 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:25.265138 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:25.265143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:25.265146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:25.265817 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:25.266066 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:25.266508 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 00:12:25.266915 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":369,"threads_started":1}
I20260811 00:12:25.270562 9584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6450d6ee5e4752a771d83b158a97a9" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45393 } }
I20260811 00:12:25.270931 9584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:25.271060 9584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6450d6ee5e4752a771d83b158a97a9, State: Initialized, Role: FOLLOWER
I20260811 00:12:25.271603 9584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6450d6ee5e4752a771d83b158a97a9" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45393 } }
I20260811 00:12:25.271842 9584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:12:25.271940 9584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:12:25.272053 9584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:25.274214 9584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6450d6ee5e4752a771d83b158a97a9" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45393 } }
I20260811 00:12:25.274708 9584 leader_election.cc:302] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a6450d6ee5e4752a771d83b158a97a9; no voters:
I20260811 00:12:25.275345 9584 leader_election.cc:288] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:12:25.275527 9594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:25.276315 9594 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6450d6ee5e4752a771d83b158a97a9, State: Running, Role: LEADER
I20260811 00:12:25.276923 9594 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6450d6ee5e4752a771d83b158a97a9" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45393 } }
I20260811 00:12:25.277422 9584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:12:25.279812 9597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a6450d6ee5e4752a771d83b158a97a9. Latest consensus state: current_term: 1 leader_uuid: "8a6450d6ee5e4752a771d83b158a97a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6450d6ee5e4752a771d83b158a97a9" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45393 } } }
I20260811 00:12:25.280284 9597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9 [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:25.280316 9595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a6450d6ee5e4752a771d83b158a97a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6450d6ee5e4752a771d83b158a97a9" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45393 } } }
I20260811 00:12:25.280879 9595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9 [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:25.283082 9603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:12:25.292524 9603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:12:25.302479 9603 catalog_manager.cc:1347] Generated new cluster ID: 014fd3f44171416f8188600c987b57ab
I20260811 00:12:25.302613 9603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:12:25.327767 9603 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:12:25.329700 9603 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:12:25.342522 9603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9: Generated new TSK 0
I20260811 00:12:25.343575 9603 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
I20260811 00:12:25.390564 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45393
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:43173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:25.727532 9631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:25.727798 9631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:25.727890 9631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:25.735023 9631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:25.735123 9631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:25.735210 9631 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:12:25.735291 9631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:12:25.745388 9631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:43173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.94.62:45393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:25.747054 9631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:25.748858 9631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:25.762490 9639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:25.762491 9641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:25.762691 9645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:25.763451 9631 server_base.cc:1034] running on GCE node
I20260811 00:12:25.764098 9631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:25.764987 9631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:25.766222 9631 hybrid_clock.cc:648] HybridClock initialized: now 1786407145766131 us; error 65 us; skew 500 ppm
I20260811 00:12:25.766518 9631 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:25 dist-test-slave-8hhm krb5kdc[9440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407145, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:25.774331 9631 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:12:25.775952 9631 webserver.cc:460] Webserver started at http://127.7.94.1:37681/ using document root <none> and password file <none>
I20260811 00:12:25.776638 9631 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:25.776767 9631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:25.777102 9631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:25.780381 9631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "6e7620e8b453422b9c5a06dd253d21f4"
format_stamp: "Formatted at 2026-08-11 00:12:25 on dist-test-slave-8hhm"
server_key: "632afefce5a42270358a5da877e1a0d5"
server_key_iv: "732419a4133437d0471791d13a447218"
server_key_version: "encryptionkey@0"
I20260811 00:12:25.781275 9631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "6e7620e8b453422b9c5a06dd253d21f4"
format_stamp: "Formatted at 2026-08-11 00:12:25 on dist-test-slave-8hhm"
server_key: "632afefce5a42270358a5da877e1a0d5"
server_key_iv: "732419a4133437d0471791d13a447218"
server_key_version: "encryptionkey@0"
I20260811 00:12:25.787130 9631 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 00:12:25.790421 9659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:25.791682 9631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:12:25.791867 9631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "6e7620e8b453422b9c5a06dd253d21f4"
format_stamp: "Formatted at 2026-08-11 00:12:25 on dist-test-slave-8hhm"
server_key: "632afefce5a42270358a5da877e1a0d5"
server_key_iv: "732419a4133437d0471791d13a447218"
server_key_version: "encryptionkey@0"
I20260811 00:12:25.792104 9631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:25.816847 9631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:25.820835 9631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:25.821137 9631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:25.823915 9631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:25.824023 9631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:25.824193 9631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:25.824290 9631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:12:25 dist-test-slave-8hhm krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407145, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:25.850342 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:25.824382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:47569 (local address 127.7.94.62:45393)
0811 00:12:25.825162 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:25.825169 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:25.826240 (+ 1071us) server_negotiation.cc:407] Connection header received
0811 00:12:25.829285 (+ 3045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:25.829292 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:25.829374 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:25.829488 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:25.831658 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:25.832958 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:25.835456 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:25.836269 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:25.836435 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:25.841147 (+ 4712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:25.841157 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:25.841165 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:25.841254 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:25.846668 (+ 5414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:25.847676 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:25.847684 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:25.847689 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:25.847833 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:25.848348 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:25.848354 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:25.848357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:25.848791 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:25.848969 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:25.849751 (+ 782us) server_negotiation.cc:299] Negotiation successful
0811 00:12:25.849988 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":539,"threads_started":1}
I20260811 00:12:25.851342 9677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:25.825087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:45393 (local address 127.7.94.1:47569)
0811 00:12:25.826048 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:25.826104 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:25.828928 (+ 2824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:25.829835 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:25.829850 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:25.830675 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:25.831448 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:25.831467 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:25.833122 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:25.833130 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:25.835300 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:25.835312 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:25.836747 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:25.836756 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:25.837098 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:25.838033 (+ 935us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:25.838073 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:25.840884 (+ 2811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:25.846814 (+ 5930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:25.846821 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:25.846832 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:25.847550 (+ 718us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:25.847948 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:25.847953 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:25.847957 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:25.848180 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:25.848955 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:25.848969 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:25.849378 (+ 409us) client_negotiation.cc:769] Sending connection context
0811 00:12:25.849668 (+ 290us) client_negotiation.cc:240] Negotiation successful
0811 00:12:25.850169 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":353,"threads_started":1}
I20260811 00:12:25.872470 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:25.857330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:41527 (local address 127.7.94.62:45393)
0811 00:12:25.857519 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:25.857525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:25.858445 (+ 920us) server_negotiation.cc:407] Connection header received
0811 00:12:25.858656 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:25.858663 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:25.858751 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:25.858850 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:25.860347 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:25.861030 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:25.863485 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:25.863956 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:25.864100 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:25.866832 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:25.866836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:25.866839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:25.866864 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:25.869346 (+ 2482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:25.870168 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:25.870173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:25.870176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:25.870340 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:25.870752 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:25.870757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:25.870760 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:25.871116 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:25.871259 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:25.871907 (+ 648us) server_negotiation.cc:299] Negotiation successful
0811 00:12:25.872125 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:12:25.872534 9770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:25.857397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:45393 (local address 127.7.94.1:41527)
0811 00:12:25.858339 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:25.858359 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:25.858516 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:25.858927 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:25.858939 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:25.859673 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:25.860154 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:25.860171 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:25.861177 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:25.861185 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:25.863300 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:25.863317 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:25.864163 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:25.864176 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:25.864456 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:25.865122 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:25.865153 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:25.866639 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:25.869460 (+ 2821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:25.869464 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:25.869468 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:25.870047 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:25.870500 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:25.870504 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:25.870507 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:25.870639 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:25.871244 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:25.871248 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:25.871492 (+ 244us) client_negotiation.cc:769] Sending connection context
0811 00:12:25.871658 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 00:12:25.872074 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"thread_start_us":575,"threads_started":1}
I20260811 00:12:25.885430 9631 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:42743
I20260811 00:12:25.885471 9853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:42743 every 8 connection(s)
I20260811 00:12:25.886606 9631 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:12:25.888784 9854 heartbeater.cc:347] Connected to a master server at 127.7.94.62:45393
I20260811 00:12:25.889014 9854 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:25.889674 9854 heartbeater.cc:511] Master 127.7.94.62:45393 requested a full tablet report, sending...
I20260811 00:12:25.891148 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 9631
I20260811 00:12:25.891404 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:12:25.891878 9511 ts_manager.cc:194] Registered new tserver with Master: 6e7620e8b453422b9c5a06dd253d21f4 (127.7.94.1:42743)
I20260811 00:12:25.892251 7544 external_mini_cluster.cc:1428] Setting key 4900d4d6cf8e085a1fa077825dcb8aff
I20260811 00:12:25.893831 9511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:47569
WARNING: no policy specified for kudu/127.7.94.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
I20260811 00:12:25.936223 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45393
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:43173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:26.253986 9860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:26.254271 9860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:26.254374 9860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:26.261541 9860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:26.261650 9860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:26.261822 9860 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:12:26.261916 9860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:12:26.271399 9860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:43173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.94.62:45393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:26.272706 9860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:26.273949 9860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:26.286895 9866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:26.286917 9868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:26.287250 9870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:26.287595 9860 server_base.cc:1034] running on GCE node
I20260811 00:12:26.288208 9860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:26.289007 9860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:26.290215 9860 hybrid_clock.cc:648] HybridClock initialized: now 1786407146290142 us; error 44 us; skew 500 ppm
I20260811 00:12:26.290495 9860 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:26 dist-test-slave-8hhm krb5kdc[9440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407146, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:26.293687 9860 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:12:26.295233 9860 webserver.cc:460] Webserver started at http://127.7.94.2:44419/ using document root <none> and password file <none>
I20260811 00:12:26.295910 9860 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:26.296038 9860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:26.296476 9860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:26.299801 9860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "d7d2e0a2066341a68280263ab5a6c013"
format_stamp: "Formatted at 2026-08-11 00:12:26 on dist-test-slave-8hhm"
server_key: "d0e535dd861c19ac672b6493faeb2acf"
server_key_iv: "f125310e69c17b61e22e9c5d4fc4db40"
server_key_version: "encryptionkey@0"
I20260811 00:12:26.300627 9860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "d7d2e0a2066341a68280263ab5a6c013"
format_stamp: "Formatted at 2026-08-11 00:12:26 on dist-test-slave-8hhm"
server_key: "d0e535dd861c19ac672b6493faeb2acf"
server_key_iv: "f125310e69c17b61e22e9c5d4fc4db40"
server_key_version: "encryptionkey@0"
I20260811 00:12:26.305722 9860 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 00:12:26.309075 9884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:26.310402 9860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:12:26.310580 9860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "d7d2e0a2066341a68280263ab5a6c013"
format_stamp: "Formatted at 2026-08-11 00:12:26 on dist-test-slave-8hhm"
server_key: "d0e535dd861c19ac672b6493faeb2acf"
server_key_iv: "f125310e69c17b61e22e9c5d4fc4db40"
server_key_version: "encryptionkey@0"
I20260811 00:12:26.310802 9860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:26.327134 9860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:26.330928 9860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:26.331200 9860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:26.333958 9860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:26.334082 9860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:26.334254 9860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:26.334329 9860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:12:26 dist-test-slave-8hhm krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407146, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:26.358081 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:26.334400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:36807 (local address 127.7.94.62:45393)
0811 00:12:26.334646 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:26.334656 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:26.336009 (+ 1353us) server_negotiation.cc:407] Connection header received
0811 00:12:26.338072 (+ 2063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:26.338081 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:26.338186 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:26.338316 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:26.340775 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:26.341894 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:26.344709 (+ 2815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:26.345137 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:26.345305 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:26.350127 (+ 4822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:26.350137 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:26.350145 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:26.350201 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:26.354430 (+ 4229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:26.355312 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:26.355320 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:26.355323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:26.355452 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:26.356064 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:26.356068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:26.356072 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:26.356477 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:26.356636 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:26.357516 (+ 880us) server_negotiation.cc:299] Negotiation successful
0811 00:12:26.357739 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 00:12:26.359238 9900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:26.334965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:45393 (local address 127.7.94.2:36807)
0811 00:12:26.335825 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:26.335878 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:26.337808 (+ 1930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:26.338629 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:26.338639 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:26.339420 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:26.340501 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:26.340536 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:26.342088 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:26.342095 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:26.344535 (+ 2440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:26.344549 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:26.345414 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:26.345422 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:26.345668 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:26.346419 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:26.346455 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:26.349890 (+ 3435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:26.354590 (+ 4700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:26.354601 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:26.354625 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:26.355192 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:26.355578 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:26.355585 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:26.355591 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:26.355934 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:26.356721 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:26.356727 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:26.357054 (+ 327us) client_negotiation.cc:769] Sending connection context
0811 00:12:26.357371 (+ 317us) client_negotiation.cc:240] Negotiation successful
0811 00:12:26.357904 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":367,"threads_started":1}
I20260811 00:12:26.380195 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:26.365381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:45393 (local address 127.7.94.2:33945)
0811 00:12:26.366212 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:26.366234 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:26.366333 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:26.366724 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:26.366727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:26.367118 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:26.367381 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:26.367388 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:26.369114 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:26.369117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:26.370561 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:26.370572 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:26.371312 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:26.371320 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:26.371483 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:26.372099 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:26.372115 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:26.372913 (+ 798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:26.377132 (+ 4219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:26.377161 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:26.377165 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:26.377851 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:26.378320 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:26.378324 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:26.378326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:26.378449 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:26.379275 (+ 826us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:26.379278 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:26.379454 (+ 176us) client_negotiation.cc:769] Sending connection context
0811 00:12:26.379592 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 00:12:26.379857 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":496,"threads_started":1}
I20260811 00:12:26.380661 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:26.365349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:33945 (local address 127.7.94.62:45393)
0811 00:12:26.365543 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:26.365550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:26.366313 (+ 763us) server_negotiation.cc:407] Connection header received
0811 00:12:26.366436 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:26.366443 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:26.366559 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:26.366695 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:26.367503 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:26.368973 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:26.370709 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:26.371123 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:26.371255 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:26.373099 (+ 1844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:26.373105 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:26.373110 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:26.373146 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:26.377009 (+ 3863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:26.377981 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:26.377990 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:26.377996 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:26.378176 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:26.378577 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:26.378584 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:26.378590 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:26.379152 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:26.379358 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:26.379855 (+ 497us) server_negotiation.cc:299] Negotiation successful
0811 00:12:26.380153 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 00:12:26.393980 9860 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:44173
I20260811 00:12:26.394068 10076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:44173 every 8 connection(s)
I20260811 00:12:26.395226 9860 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:12:26.397261 10077 heartbeater.cc:347] Connected to a master server at 127.7.94.62:45393
I20260811 00:12:26.397456 10077 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:26.398054 10077 heartbeater.cc:511] Master 127.7.94.62:45393 requested a full tablet report, sending...
I20260811 00:12:26.399353 9511 ts_manager.cc:194] Registered new tserver with Master: d7d2e0a2066341a68280263ab5a6c013 (127.7.94.2:44173)
I20260811 00:12:26.400110 9511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:36807
I20260811 00:12:26.404096 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 9860
I20260811 00:12:26.404261 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:12:26.404609 7544 external_mini_cluster.cc:1428] Setting key facf1ff7ac3633864d014eb9d0c100e5
WARNING: no policy specified for kudu/127.7.94.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
I20260811 00:12:26.448479 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45393
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:43173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:26.768220 10084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:26.768483 10084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:26.768636 10084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:26.775754 10084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:26.775874 10084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:26.776015 10084 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:12:26.776139 10084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:12:26.785818 10084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:43173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.94.62:45393
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:26.787109 10084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:26.788432 10084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:26.802311 10092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:26.802273 10091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:26.802354 10094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:26.803149 10084 server_base.cc:1034] running on GCE node
I20260811 00:12:26.803757 10084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:26.804649 10084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:26.805892 10084 hybrid_clock.cc:648] HybridClock initialized: now 1786407146805832 us; error 32 us; skew 500 ppm
I20260811 00:12:26.806186 10084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:26 dist-test-slave-8hhm krb5kdc[9440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407146, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:26.809319 10084 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:12:26.810725 10084 webserver.cc:460] Webserver started at http://127.7.94.3:37703/ using document root <none> and password file <none>
I20260811 00:12:26.811381 10084 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:26.811508 10084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:26.811863 10084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:26.814909 10084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "bd92062178ec4d9ca39fd0a82d996b38"
format_stamp: "Formatted at 2026-08-11 00:12:26 on dist-test-slave-8hhm"
server_key: "454f9bcf816a0548ae0e5c434e9b9dcc"
server_key_iv: "3f2644a4e5e30ee70eaf5bb11255cf95"
server_key_version: "encryptionkey@0"
I20260811 00:12:26.815812 10084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "bd92062178ec4d9ca39fd0a82d996b38"
format_stamp: "Formatted at 2026-08-11 00:12:26 on dist-test-slave-8hhm"
server_key: "454f9bcf816a0548ae0e5c434e9b9dcc"
server_key_iv: "3f2644a4e5e30ee70eaf5bb11255cf95"
server_key_version: "encryptionkey@0"
I20260811 00:12:26.820749 10084 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:12:26.824270 10105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:26.825500 10084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:12:26.825688 10084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "bd92062178ec4d9ca39fd0a82d996b38"
format_stamp: "Formatted at 2026-08-11 00:12:26 on dist-test-slave-8hhm"
server_key: "454f9bcf816a0548ae0e5c434e9b9dcc"
server_key_iv: "3f2644a4e5e30ee70eaf5bb11255cf95"
server_key_version: "encryptionkey@0"
I20260811 00:12:26.825917 10084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:26.837564 10084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:26.841151 10084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:26.841441 10084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:26.844015 10084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:26.844136 10084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:26.844298 10084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:26.844360 10084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:12:26 dist-test-slave-8hhm krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407146, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:26.867419 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:26.844344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:47941 (local address 127.7.94.62:45393)
0811 00:12:26.844558 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:26.844564 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:26.845855 (+ 1291us) server_negotiation.cc:407] Connection header received
0811 00:12:26.847900 (+ 2045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:26.847906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:26.847996 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:26.848100 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:26.850140 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:26.851492 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:26.854137 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:26.854725 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:26.854869 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:26.860223 (+ 5354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:26.860227 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:26.860231 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:26.860261 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:26.863868 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:26.864859 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:26.864865 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:26.864868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:26.865003 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:26.865511 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:26.865517 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:26.865520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:26.865943 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:26.866115 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:26.866816 (+ 701us) server_negotiation.cc:299] Negotiation successful
0811 00:12:26.867048 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 00:12:26.868373 10121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:26.844778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:45393 (local address 127.7.94.3:47941)
0811 00:12:26.845668 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:26.845733 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:26.847619 (+ 1886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:26.848343 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:26.848351 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:26.849079 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:26.849920 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:26.849959 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:26.851685 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:26.851689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:26.853892 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:26.853904 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:26.855017 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:26.855028 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:26.855392 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:26.856283 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:26.856328 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:26.860005 (+ 3677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:26.864016 (+ 4011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:26.864025 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:26.864039 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:26.864724 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:26.865112 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:26.865117 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:26.865121 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:26.865391 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:26.866075 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:26.866086 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:26.866455 (+ 369us) client_negotiation.cc:769] Sending connection context
0811 00:12:26.866707 (+ 252us) client_negotiation.cc:240] Negotiation successful
0811 00:12:26.867239 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":406,"threads_started":1}
I20260811 00:12:26.889492 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:26.874312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:53251 (local address 127.7.94.62:45393)
0811 00:12:26.874543 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:26.874552 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:26.875178 (+ 626us) server_negotiation.cc:407] Connection header received
0811 00:12:26.875302 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:26.875309 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:26.875403 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:26.875532 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:26.876394 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:26.877800 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:26.880349 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:26.880900 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:26.881024 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:26.883144 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:26.883150 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:26.883154 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:26.883183 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:26.886270 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:26.887202 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:26.887208 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:26.887212 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:26.887366 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:26.887834 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:26.887842 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:26.887846 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:26.888168 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:26.888287 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:26.888899 (+ 612us) server_negotiation.cc:299] Negotiation successful
0811 00:12:26.889123 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:12:26.890486 10224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:26.874407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:45393 (local address 127.7.94.3:53251)
0811 00:12:26.875074 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:26.875092 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:26.875209 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:26.875539 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:26.875542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:26.875959 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:26.876235 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:26.876244 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:26.877966 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:26.877973 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:26.880129 (+ 2156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:26.880176 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:26.881087 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:26.881094 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:26.881348 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:26.882132 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:26.882154 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:26.882951 (+ 797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:26.886489 (+ 3538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:26.886497 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:26.886503 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:26.887067 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:26.887518 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:26.887523 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:26.887526 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:26.887700 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:26.888402 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:26.888407 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:26.888590 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 00:12:26.889738 (+ 1148us) client_negotiation.cc:240] Negotiation successful
0811 00:12:26.890049 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":375,"threads_started":1}
I20260811 00:12:26.897228 9854 heartbeater.cc:503] Master 127.7.94.62:45393 was elected leader, sending a full tablet report...
I20260811 00:12:26.907805 10084 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:37631
I20260811 00:12:26.907840 10310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:37631 every 8 connection(s)
I20260811 00:12:26.908948 10084 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:12:26.910823 10313 heartbeater.cc:347] Connected to a master server at 127.7.94.62:45393
I20260811 00:12:26.911029 10313 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:26.911614 10313 heartbeater.cc:511] Master 127.7.94.62:45393 requested a full tablet report, sending...
I20260811 00:12:26.912854 9511 ts_manager.cc:194] Registered new tserver with Master: bd92062178ec4d9ca39fd0a82d996b38 (127.7.94.3:37631)
I20260811 00:12:26.913637 9511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:47941
I20260811 00:12:26.917080 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 10084
I20260811 00:12:26.917294 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:12:26.917676 7544 external_mini_cluster.cc:1428] Setting key 6f65b1e5ab402f628424766964b1b7e6
I20260811 00:12:26.920240 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 00:12:26 dist-test-slave-8hhm krb5kdc[9440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407146, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 00:12:26 dist-test-slave-8hhm krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407146, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:26.970216 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:26.956595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59850 (local address 127.7.94.62:45393)
0811 00:12:26.956820 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:26.956827 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:26.957025 (+ 198us) server_negotiation.cc:407] Connection header received
0811 00:12:26.957176 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:26.957180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:26.957290 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:26.957383 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:26.958714 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:26.959432 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:26.960777 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:26.961192 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:26.961358 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:26.965090 (+ 3732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:26.965099 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:26.965106 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:26.965155 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:26.967604 (+ 2449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:26.968200 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:26.968206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:26.968210 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:26.968336 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:26.968709 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:26.968714 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:26.968717 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:26.969084 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:26.969346 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:26.969715 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 00:12:26.969891 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 00:12:26.974102 9511 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:12:26.980960 9511 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:12:27.004257 10336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:26.992521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:37631 (local address 127.0.0.1:33402)
0811 00:12:26.993534 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:26.993553 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:26.993671 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:26.995193 (+ 1522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:26.995200 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:26.995230 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:26.995632 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:26.995645 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:26.997986 (+ 2341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:26.997995 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.001258 (+ 3263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:27.001273 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.003148 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.003160 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.003429 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.003510 (+ 81us) client_negotiation.cc:769] Sending connection context
0811 00:12:27.003637 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 00:12:27.003819 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":893,"mutex_wait_us":45,"thread_start_us":557,"threads_started":1}
I20260811 00:12:27.005277 10338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:26.993732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33402 (local address 127.7.94.3:37631)
0811 00:12:26.994547 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:26.994554 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:26.994582 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:12:26.994658 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:26.994664 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:26.994837 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:26.995867 (+ 1030us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:26.996223 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:26.997848 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.001439 (+ 3591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.002926 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.003060 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.003287 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:27.004757 (+ 1470us) server_negotiation.cc:299] Negotiation successful
0811 00:12:27.004932 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":428,"threads_started":1}
I20260811 00:12:27.005883 10335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:26.992442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:44173 (local address 127.0.0.1:60852)
0811 00:12:26.993339 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:26.993391 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:26.993553 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:26.998849 (+ 5296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:26.998858 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:26.998895 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:26.999474 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:26.999485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.001310 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.001318 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.003485 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:27.003497 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.005125 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.005131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.005386 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.005430 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 00:12:27.005508 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 00:12:27.005614 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"mutex_wait_us":69,"thread_start_us":1316,"threads_started":1}
I20260811 00:12:27.006634 10337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:26.992921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60852 (local address 127.7.94.2:44173)
0811 00:12:26.998378 (+ 5457us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:26.998384 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:26.998406 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:12:26.998488 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:26.998494 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:26.998662 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:26.998800 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:26.999700 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.001177 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.003624 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.004667 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.004792 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.004993 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:27.006165 (+ 1172us) server_negotiation.cc:299] Negotiation successful
0811 00:12:27.006356 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5322,"thread_start_us":5209,"threads_started":1}
I20260811 00:12:27.007014 10339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:26.994017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:42743 (local address 127.0.0.1:34290)
0811 00:12:26.999464 (+ 5447us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:26.999482 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:26.999611 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:27.000434 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:27.000438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:27.000461 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:27.000883 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:27.000894 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.002284 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.002289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.004207 (+ 1918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:27.004224 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.006288 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.006294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.006456 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.006488 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 00:12:27.006566 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 00:12:27.006692 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5291,"mutex_wait_us":41,"thread_start_us":5102,"threads_started":1}
I20260811 00:12:27.008430 10341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:26.994651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34290 (local address 127.7.94.1:42743)
0811 00:12:26.995305 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:26.995315 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:26.999602 (+ 4287us) server_negotiation.cc:407] Connection header received
0811 00:12:26.999763 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:26.999772 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:26.999986 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:27.000291 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:27.001169 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.002149 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.004652 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.006070 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.007712 (+ 1642us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.007904 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:27.007972 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 00:12:27.008159 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":295,"threads_started":1}
I20260811 00:12:27.009166 10188 tablet_service.cc:1467] Processing CreateTablet for tablet 08ef16bab4ce4ca598a406dbf7271d01 (DEFAULT_TABLE table=test-table [id=cad680c8d01948f3905b7dfe619cd666]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:27.009881 9963 tablet_service.cc:1467] Processing CreateTablet for tablet 08ef16bab4ce4ca598a406dbf7271d01 (DEFAULT_TABLE table=test-table [id=cad680c8d01948f3905b7dfe619cd666]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:27.010790 10188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08ef16bab4ce4ca598a406dbf7271d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:27.011058 9963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08ef16bab4ce4ca598a406dbf7271d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:27.011686 9745 tablet_service.cc:1467] Processing CreateTablet for tablet 08ef16bab4ce4ca598a406dbf7271d01 (DEFAULT_TABLE table=test-table [id=cad680c8d01948f3905b7dfe619cd666]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:27.012979 9745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08ef16bab4ce4ca598a406dbf7271d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:27.021888 10342 tablet_bootstrap.cc:492] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013: Bootstrap starting.
I20260811 00:12:27.025352 10343 tablet_bootstrap.cc:492] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38: Bootstrap starting.
I20260811 00:12:27.026202 10342 tablet_bootstrap.cc:654] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:27.027110 10344 tablet_bootstrap.cc:492] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4: Bootstrap starting.
I20260811 00:12:27.027480 10342 log.cc:824] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:27.028640 10343 tablet_bootstrap.cc:654] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:27.029918 10343 log.cc:824] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:27.030685 10344 tablet_bootstrap.cc:654] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:27.030802 10342 tablet_bootstrap.cc:492] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013: No bootstrap required, opened a new log
I20260811 00:12:27.031528 10342 ts_tablet_manager.cc:1397] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.002s
I20260811 00:12:27.031867 10344 log.cc:824] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:27.034458 10344 tablet_bootstrap.cc:492] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4: No bootstrap required, opened a new log
I20260811 00:12:27.034704 10344 ts_tablet_manager.cc:1397] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 00:12:27.039130 10343 tablet_bootstrap.cc:492] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38: No bootstrap required, opened a new log
I20260811 00:12:27.039388 10343 ts_tablet_manager.cc:1397] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.000s
I20260811 00:12:27.039793 10342 raft_consensus.cc:348] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } } peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } }
I20260811 00:12:27.040221 10342 raft_consensus.cc:374] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:27.040325 10342 raft_consensus.cc:729] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7d2e0a2066341a68280263ab5a6c013, State: Initialized, Role: FOLLOWER
I20260811 00:12:27.040920 10342 consensus_queue.cc:260] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } } peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } }
I20260811 00:12:27.042133 10344 raft_consensus.cc:348] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } } peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } }
I20260811 00:12:27.042536 10344 raft_consensus.cc:374] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:27.042650 10344 raft_consensus.cc:729] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e7620e8b453422b9c5a06dd253d21f4, State: Initialized, Role: FOLLOWER
I20260811 00:12:27.043265 10077 heartbeater.cc:503] Master 127.7.94.62:45393 was elected leader, sending a full tablet report...
I20260811 00:12:27.043323 10344 consensus_queue.cc:260] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } } peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } }
I20260811 00:12:27.043843 10342 ts_tablet_manager.cc:1428] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013: Time spent starting tablet: real 0.012s user 0.010s sys 0.001s
I20260811 00:12:27.045827 10344 ts_tablet_manager.cc:1428] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4: Time spent starting tablet: real 0.011s user 0.003s sys 0.008s
I20260811 00:12:27.049726 10343 raft_consensus.cc:348] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } } peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } }
I20260811 00:12:27.050120 10343 raft_consensus.cc:374] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:27.050222 10343 raft_consensus.cc:729] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd92062178ec4d9ca39fd0a82d996b38, State: Initialized, Role: FOLLOWER
I20260811 00:12:27.050688 10343 consensus_queue.cc:260] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } } peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } }
I20260811 00:12:27.051604 10313 heartbeater.cc:503] Master 127.7.94.62:45393 was elected leader, sending a full tablet report...
I20260811 00:12:27.052218 10343 ts_tablet_manager.cc:1428] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260811 00:12:27.124234 10352 raft_consensus.cc:482] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:12:27.124577 10352 raft_consensus.cc:504] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } } peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } }
I20260811 00:12:27.126416 10352 leader_election.cc:288] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7d2e0a2066341a68280263ab5a6c013 (127.7.94.2:44173), 6e7620e8b453422b9c5a06dd253d21f4 (127.7.94.1:42743)
I20260811 00:12:27.134140 10337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.126769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:46229 (local address 127.7.94.2:44173)
0811 00:12:27.127019 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:27.127028 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:27.127414 (+ 386us) server_negotiation.cc:407] Connection header received
0811 00:12:27.127675 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:27.127679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:27.127742 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:27.127841 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:27.128358 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.129531 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.132071 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.133276 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.133420 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.133529 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:27.133744 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 00:12:27.133894 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 00:12:27.134159 10121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.126901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:44173 (local address 127.7.94.3:46229)
0811 00:12:27.127220 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:27.127248 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:27.127507 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:27.127870 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:27.127878 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:27.127925 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:27.128179 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:27.128193 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.129676 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.129681 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.131886 (+ 2205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:27.131911 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.133423 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.133427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.133552 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.133574 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 00:12:27.133695 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 00:12:27.133834 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":29}
I20260811 00:12:27.134768 10341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.126962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:38825 (local address 127.7.94.1:42743)
0811 00:12:27.127201 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:27.127208 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:27.128169 (+ 961us) server_negotiation.cc:407] Connection header received
0811 00:12:27.128314 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:27.128322 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:27.128404 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:27.128482 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:27.129169 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.130122 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.132441 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.133746 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.133884 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.133976 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:27.134341 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 00:12:27.134491 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 00:12:27.135387 10353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.126950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:42743 (local address 127.7.94.3:38825)
0811 00:12:27.128052 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:27.128068 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:27.128187 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:27.128518 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:27.128523 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:27.128548 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:27.128939 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:27.128951 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.130327 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.130337 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.132289 (+ 1952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:27.132300 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.133966 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.133972 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.134139 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.134175 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 00:12:27.134857 (+ 682us) client_negotiation.cc:240] Negotiation successful
0811 00:12:27.134995 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":938,"mutex_wait_us":1,"thread_start_us":719,"threads_started":1}
I20260811 00:12:27.135262 10005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08ef16bab4ce4ca598a406dbf7271d01" candidate_uuid: "bd92062178ec4d9ca39fd0a82d996b38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7d2e0a2066341a68280263ab5a6c013" is_pre_election: true
I20260811 00:12:27.135802 10005 raft_consensus.cc:2454] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd92062178ec4d9ca39fd0a82d996b38 in term 0.
I20260811 00:12:27.136353 9781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08ef16bab4ce4ca598a406dbf7271d01" candidate_uuid: "bd92062178ec4d9ca39fd0a82d996b38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7620e8b453422b9c5a06dd253d21f4" is_pre_election: true
I20260811 00:12:27.136507 10107 leader_election.cc:302] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd92062178ec4d9ca39fd0a82d996b38, d7d2e0a2066341a68280263ab5a6c013; no voters:
I20260811 00:12:27.136826 9781 raft_consensus.cc:2454] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd92062178ec4d9ca39fd0a82d996b38 in term 0.
I20260811 00:12:27.136953 10352 raft_consensus.cc:2781] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:12:27.137127 10352 raft_consensus.cc:482] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:12:27.137270 10352 raft_consensus.cc:3037] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:27.139555 10352 raft_consensus.cc:504] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } } peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } }
W20260811 00:12:27.140074 9855 tablet.cc:2369] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:12:27.140373 10352 leader_election.cc:288] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [CANDIDATE]: Term 1 election: Requested vote from peers d7d2e0a2066341a68280263ab5a6c013 (127.7.94.2:44173), 6e7620e8b453422b9c5a06dd253d21f4 (127.7.94.1:42743)
I20260811 00:12:27.140772 10005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08ef16bab4ce4ca598a406dbf7271d01" candidate_uuid: "bd92062178ec4d9ca39fd0a82d996b38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7d2e0a2066341a68280263ab5a6c013"
I20260811 00:12:27.140852 9781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08ef16bab4ce4ca598a406dbf7271d01" candidate_uuid: "bd92062178ec4d9ca39fd0a82d996b38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7620e8b453422b9c5a06dd253d21f4"
I20260811 00:12:27.140990 10005 raft_consensus.cc:3037] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:27.141100 9781 raft_consensus.cc:3037] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:27.143445 9781 raft_consensus.cc:2454] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd92062178ec4d9ca39fd0a82d996b38 in term 1.
I20260811 00:12:27.143560 10005 raft_consensus.cc:2454] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd92062178ec4d9ca39fd0a82d996b38 in term 1.
I20260811 00:12:27.143836 10108 leader_election.cc:302] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e7620e8b453422b9c5a06dd253d21f4, bd92062178ec4d9ca39fd0a82d996b38; no voters:
I20260811 00:12:27.144119 10352 raft_consensus.cc:2781] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:27.144630 10352 raft_consensus.cc:686] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [term 1 LEADER]: Becoming Leader. State: Replica: bd92062178ec4d9ca39fd0a82d996b38, State: Running, Role: LEADER
I20260811 00:12:27.145081 10352 consensus_queue.cc:237] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } } peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } }
W20260811 00:12:27.147269 10078 tablet.cc:2369] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:12:27.149791 9510 catalog_manager.cc:5495] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd92062178ec4d9ca39fd0a82d996b38 (127.7.94.3). New cstate: current_term: 1 leader_uuid: "bd92062178ec4d9ca39fd0a82d996b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } health_report { overall_health: UNKNOWN } } }
W20260811 00:12:27.160493 10315 tablet.cc:2369] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:12:27.198655 10338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.192951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33410 (local address 127.7.94.3:37631)
0811 00:12:27.193175 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:27.193184 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:27.193293 (+ 109us) server_negotiation.cc:407] Connection header received
0811 00:12:27.193563 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:27.193566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:27.193661 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:27.193775 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:12:27.194074 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.194850 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.196666 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.197027 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.197117 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.197679 (+ 562us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:12:27.198098 (+ 419us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:12:27.198222 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:27.198306 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 00:12:27.198364 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 00:12:27.206043 9781 raft_consensus.cc:1261] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4 [term 1 FOLLOWER]: Refusing update from remote peer bd92062178ec4d9ca39fd0a82d996b38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:12:27.206133 10005 raft_consensus.cc:1261] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013 [term 1 FOLLOWER]: Refusing update from remote peer bd92062178ec4d9ca39fd0a82d996b38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:12:27.206995 10354 consensus_queue.cc:1035] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:12:27.207314 10352 consensus_queue.cc:1035] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:12:27.218748 10360 mvcc.cc:205] Tried to move back new op lower bound from 7317123674946875392 to 7317123674710257664. Current Snapshot: MvccSnapshot[applied={T|T < 7317123674946875392}]
I20260811 00:12:27.239369 9510 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:12:27.239712 9510 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:12:27.243316 9510 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9: Sending DeleteTablet for 3 replicas of tablet 08ef16bab4ce4ca598a406dbf7271d01
I20260811 00:12:27.244488 9963 tablet_service.cc:1514] Processing DeleteTablet for tablet 08ef16bab4ce4ca598a406dbf7271d01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:27 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:60852
I20260811 00:12:27.245124 10188 tablet_service.cc:1514] Processing DeleteTablet for tablet 08ef16bab4ce4ca598a406dbf7271d01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:27 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:33402
I20260811 00:12:27.246166 9745 tablet_service.cc:1514] Processing DeleteTablet for tablet 08ef16bab4ce4ca598a406dbf7271d01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:27 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:34290
I20260811 00:12:27.253311 10383 tablet_replica.cc:330] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38: stopping tablet replica
I20260811 00:12:27.253717 10383 raft_consensus.cc:2229] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:12:27.254338 10383 raft_consensus.cc:2258] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:27.254226 9510 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:12:27.255056 9510 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:12:27.256907 10384 tablet_replica.cc:330] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4: stopping tablet replica
I20260811 00:12:27.257367 10384 raft_consensus.cc:2229] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:12:27.257915 10384 raft_consensus.cc:2258] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:27.258714 10381 tablet_replica.cc:330] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013: stopping tablet replica
I20260811 00:12:27.259122 10381 raft_consensus.cc:2229] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:12:27.259657 10381 raft_consensus.cc:2258] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:27.263394 10381 ts_tablet_manager.cc:1905] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:27.264770 10384 ts_tablet_manager.cc:1905] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:27.264935 10188 tablet_service.cc:1467] Processing CreateTablet for tablet 60962ad60de54b0f85a190f335c8234f (DEFAULT_TABLE table=test-table [id=4035369e472842cc94911072bfe46508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:27.265465 10188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60962ad60de54b0f85a190f335c8234f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:27.266230 10383 ts_tablet_manager.cc:1905] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:27.271145 9963 tablet_service.cc:1467] Processing CreateTablet for tablet 60962ad60de54b0f85a190f335c8234f (DEFAULT_TABLE table=test-table [id=4035369e472842cc94911072bfe46508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:27.271575 9963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60962ad60de54b0f85a190f335c8234f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:27.272002 10384 ts_tablet_manager.cc:1918] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:12:27.272172 10384 log.cc:1196] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/08ef16bab4ce4ca598a406dbf7271d01
I20260811 00:12:27.272761 10384 ts_tablet_manager.cc:1939] T 08ef16bab4ce4ca598a406dbf7271d01 P 6e7620e8b453422b9c5a06dd253d21f4: Deleting consensus metadata
I20260811 00:12:27.274659 9490 catalog_manager.cc:4841] TS 6e7620e8b453422b9c5a06dd253d21f4 (127.7.94.1:42743): tablet 08ef16bab4ce4ca598a406dbf7271d01 (table test-table [id=cad680c8d01948f3905b7dfe619cd666]) successfully deleted
I20260811 00:12:27.276506 9745 tablet_service.cc:1467] Processing CreateTablet for tablet 60962ad60de54b0f85a190f335c8234f (DEFAULT_TABLE table=test-table [id=4035369e472842cc94911072bfe46508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:27.276906 10383 ts_tablet_manager.cc:1918] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:12:27.276950 9745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60962ad60de54b0f85a190f335c8234f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:27.277168 10383 log.cc:1196] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/08ef16bab4ce4ca598a406dbf7271d01
I20260811 00:12:27.277464 10381 ts_tablet_manager.cc:1918] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:12:27.277630 10381 log.cc:1196] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/08ef16bab4ce4ca598a406dbf7271d01
I20260811 00:12:27.278024 10383 ts_tablet_manager.cc:1939] T 08ef16bab4ce4ca598a406dbf7271d01 P bd92062178ec4d9ca39fd0a82d996b38: Deleting consensus metadata
I20260811 00:12:27.278172 10381 ts_tablet_manager.cc:1939] T 08ef16bab4ce4ca598a406dbf7271d01 P d7d2e0a2066341a68280263ab5a6c013: Deleting consensus metadata
I20260811 00:12:27.279812 9490 catalog_manager.cc:4841] TS bd92062178ec4d9ca39fd0a82d996b38 (127.7.94.3:37631): tablet 08ef16bab4ce4ca598a406dbf7271d01 (table test-table [id=cad680c8d01948f3905b7dfe619cd666]) successfully deleted
I20260811 00:12:27.280197 9489 catalog_manager.cc:4841] TS d7d2e0a2066341a68280263ab5a6c013 (127.7.94.2:44173): tablet 08ef16bab4ce4ca598a406dbf7271d01 (table test-table [id=cad680c8d01948f3905b7dfe619cd666]) successfully deleted
I20260811 00:12:27.283069 10343 tablet_bootstrap.cc:492] T 60962ad60de54b0f85a190f335c8234f P bd92062178ec4d9ca39fd0a82d996b38: Bootstrap starting.
I20260811 00:12:27.283988 10342 tablet_bootstrap.cc:492] T 60962ad60de54b0f85a190f335c8234f P d7d2e0a2066341a68280263ab5a6c013: Bootstrap starting.
I20260811 00:12:27.284776 10343 tablet_bootstrap.cc:654] T 60962ad60de54b0f85a190f335c8234f P bd92062178ec4d9ca39fd0a82d996b38: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:27.285616 10342 tablet_bootstrap.cc:654] T 60962ad60de54b0f85a190f335c8234f P d7d2e0a2066341a68280263ab5a6c013: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:27.287748 10344 tablet_bootstrap.cc:492] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4: Bootstrap starting.
I20260811 00:12:27.289877 10344 tablet_bootstrap.cc:654] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:27.296674 10344 tablet_bootstrap.cc:492] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4: No bootstrap required, opened a new log
I20260811 00:12:27.296834 10344 ts_tablet_manager.cc:1397] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 00:12:27.297113 10342 tablet_bootstrap.cc:492] T 60962ad60de54b0f85a190f335c8234f P d7d2e0a2066341a68280263ab5a6c013: No bootstrap required, opened a new log
I20260811 00:12:27.297330 10342 ts_tablet_manager.cc:1397] T 60962ad60de54b0f85a190f335c8234f P d7d2e0a2066341a68280263ab5a6c013: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20260811 00:12:27.297875 10343 tablet_bootstrap.cc:492] T 60962ad60de54b0f85a190f335c8234f P bd92062178ec4d9ca39fd0a82d996b38: No bootstrap required, opened a new log
I20260811 00:12:27.297748 10344 raft_consensus.cc:348] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } }
I20260811 00:12:27.297971 10344 raft_consensus.cc:374] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:27.298029 10343 ts_tablet_manager.cc:1397] T 60962ad60de54b0f85a190f335c8234f P bd92062178ec4d9ca39fd0a82d996b38: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260811 00:12:27.298079 10344 raft_consensus.cc:729] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e7620e8b453422b9c5a06dd253d21f4, State: Initialized, Role: FOLLOWER
I20260811 00:12:27.298238 10342 raft_consensus.cc:348] T 60962ad60de54b0f85a190f335c8234f P d7d2e0a2066341a68280263ab5a6c013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } }
I20260811 00:12:27.298465 10342 raft_consensus.cc:374] T 60962ad60de54b0f85a190f335c8234f P d7d2e0a2066341a68280263ab5a6c013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:27.298568 10342 raft_consensus.cc:729] T 60962ad60de54b0f85a190f335c8234f P d7d2e0a2066341a68280263ab5a6c013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7d2e0a2066341a68280263ab5a6c013, State: Initialized, Role: FOLLOWER
I20260811 00:12:27.298907 10342 consensus_queue.cc:260] T 60962ad60de54b0f85a190f335c8234f P d7d2e0a2066341a68280263ab5a6c013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } }
I20260811 00:12:27.298964 10343 raft_consensus.cc:348] T 60962ad60de54b0f85a190f335c8234f P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } }
I20260811 00:12:27.299173 10343 raft_consensus.cc:374] T 60962ad60de54b0f85a190f335c8234f P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:27.299269 10343 raft_consensus.cc:729] T 60962ad60de54b0f85a190f335c8234f P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd92062178ec4d9ca39fd0a82d996b38, State: Initialized, Role: FOLLOWER
I20260811 00:12:27.299532 10342 ts_tablet_manager.cc:1428] T 60962ad60de54b0f85a190f335c8234f P d7d2e0a2066341a68280263ab5a6c013: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 00:12:27.299512 10344 consensus_queue.cc:260] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } }
I20260811 00:12:27.299582 10343 consensus_queue.cc:260] T 60962ad60de54b0f85a190f335c8234f P bd92062178ec4d9ca39fd0a82d996b38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } }
I20260811 00:12:27.300264 10343 ts_tablet_manager.cc:1428] T 60962ad60de54b0f85a190f335c8234f P bd92062178ec4d9ca39fd0a82d996b38: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 00:12:27.300969 10344 ts_tablet_manager.cc:1428] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 00:12:27.358748 10351 raft_consensus.cc:482] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:12:27.358990 10351 raft_consensus.cc:504] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } }
I20260811 00:12:27.360714 10351 leader_election.cc:288] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7d2e0a2066341a68280263ab5a6c013 (127.7.94.2:44173), bd92062178ec4d9ca39fd0a82d996b38 (127.7.94.3:37631)
I20260811 00:12:27.368850 10338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.361464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:60647 (local address 127.7.94.3:37631)
0811 00:12:27.361641 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:27.361657 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:27.361987 (+ 330us) server_negotiation.cc:407] Connection header received
0811 00:12:27.362366 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:27.362374 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:27.362444 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:27.362541 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:27.363276 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.364457 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.366886 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.367725 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.367871 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.367984 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:27.368346 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 00:12:27.368489 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:12:27.369138 10337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.361136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:52649 (local address 127.7.94.2:44173)
0811 00:12:27.361362 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:27.361371 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:27.362257 (+ 886us) server_negotiation.cc:407] Connection header received
0811 00:12:27.362390 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:27.362396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:27.362475 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:27.362567 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:27.363408 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.364182 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.366872 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.368128 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.368259 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.368353 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:27.368701 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 00:12:27.368859 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 00:12:27.369192 10388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.361084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:44173 (local address 127.7.94.1:52649)
0811 00:12:27.362160 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:27.362175 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:27.362276 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:27.362744 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:27.362750 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:27.362790 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:27.363245 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:27.363257 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.364313 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.364319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.366686 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:27.366703 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.368350 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.368357 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.368514 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.368546 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 00:12:27.368633 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 00:12:27.368766 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":971,"mutex_wait_us":56,"thread_start_us":447,"threads_started":1}
I20260811 00:12:27.369771 10389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.361033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:37631 (local address 127.7.94.1:60647)
0811 00:12:27.361843 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:27.361869 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:27.362154 (+ 285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:27.362570 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:27.362574 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:27.362609 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:27.363064 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:27.363082 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.364638 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.364643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.366757 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:27.366772 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.367969 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.367976 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.368131 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.368155 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 00:12:27.369027 (+ 872us) client_negotiation.cc:240] Negotiation successful
0811 00:12:27.369167 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"mutex_wait_us":1,"thread_start_us":862,"threads_started":1}
I20260811 00:12:27.369999 10005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60962ad60de54b0f85a190f335c8234f" candidate_uuid: "6e7620e8b453422b9c5a06dd253d21f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7d2e0a2066341a68280263ab5a6c013" is_pre_election: true
I20260811 00:12:27.370283 10005 raft_consensus.cc:2454] T 60962ad60de54b0f85a190f335c8234f P d7d2e0a2066341a68280263ab5a6c013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e7620e8b453422b9c5a06dd253d21f4 in term 0.
I20260811 00:12:27.370812 9661 leader_election.cc:302] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e7620e8b453422b9c5a06dd253d21f4, d7d2e0a2066341a68280263ab5a6c013; no voters:
I20260811 00:12:27.370877 10233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60962ad60de54b0f85a190f335c8234f" candidate_uuid: "6e7620e8b453422b9c5a06dd253d21f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd92062178ec4d9ca39fd0a82d996b38" is_pre_election: true
I20260811 00:12:27.371258 10351 raft_consensus.cc:2781] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:12:27.371381 10351 raft_consensus.cc:482] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:12:27.371308 10233 raft_consensus.cc:2454] T 60962ad60de54b0f85a190f335c8234f P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e7620e8b453422b9c5a06dd253d21f4 in term 0.
I20260811 00:12:27.371487 10351 raft_consensus.cc:3037] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:27.374133 10351 raft_consensus.cc:504] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } }
I20260811 00:12:27.374846 10351 leader_election.cc:288] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [CANDIDATE]: Term 1 election: Requested vote from peers d7d2e0a2066341a68280263ab5a6c013 (127.7.94.2:44173), bd92062178ec4d9ca39fd0a82d996b38 (127.7.94.3:37631)
I20260811 00:12:27.375222 10005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60962ad60de54b0f85a190f335c8234f" candidate_uuid: "6e7620e8b453422b9c5a06dd253d21f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7d2e0a2066341a68280263ab5a6c013"
I20260811 00:12:27.375348 10233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60962ad60de54b0f85a190f335c8234f" candidate_uuid: "6e7620e8b453422b9c5a06dd253d21f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd92062178ec4d9ca39fd0a82d996b38"
I20260811 00:12:27.375443 10005 raft_consensus.cc:3037] T 60962ad60de54b0f85a190f335c8234f P d7d2e0a2066341a68280263ab5a6c013 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:27.375571 10233 raft_consensus.cc:3037] T 60962ad60de54b0f85a190f335c8234f P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:27.378556 10005 raft_consensus.cc:2454] T 60962ad60de54b0f85a190f335c8234f P d7d2e0a2066341a68280263ab5a6c013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e7620e8b453422b9c5a06dd253d21f4 in term 1.
I20260811 00:12:27.378937 9661 leader_election.cc:302] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e7620e8b453422b9c5a06dd253d21f4, d7d2e0a2066341a68280263ab5a6c013; no voters:
I20260811 00:12:27.378957 10233 raft_consensus.cc:2454] T 60962ad60de54b0f85a190f335c8234f P bd92062178ec4d9ca39fd0a82d996b38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e7620e8b453422b9c5a06dd253d21f4 in term 1.
I20260811 00:12:27.379227 10351 raft_consensus.cc:2781] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:27.379719 10351 raft_consensus.cc:686] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [term 1 LEADER]: Becoming Leader. State: Replica: 6e7620e8b453422b9c5a06dd253d21f4, State: Running, Role: LEADER
I20260811 00:12:27.380128 10351 consensus_queue.cc:237] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } }
I20260811 00:12:27.383759 9509 catalog_manager.cc:5495] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e7620e8b453422b9c5a06dd253d21f4 (127.7.94.1). New cstate: current_term: 1 leader_uuid: "6e7620e8b453422b9c5a06dd253d21f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } health_report { overall_health: UNKNOWN } } }
Aug 11 00:12:27 dist-test-slave-8hhm krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407145, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:27.442689 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.427885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59852 (local address 127.7.94.62:45393)
0811 00:12:27.428143 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:27.428152 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:27.428639 (+ 487us) server_negotiation.cc:407] Connection header received
0811 00:12:27.428835 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:27.428838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:27.428906 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:27.429030 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:27.430400 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.431136 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.432402 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.432835 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.432975 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.436382 (+ 3407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:27.436388 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:27.436394 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:27.436429 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:27.439887 (+ 3458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:27.440577 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:27.440583 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:27.440586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:27.440748 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:27.441132 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:27.441137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:27.441141 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:27.441508 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:27.441647 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:27.442044 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 00:12:27.442372 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":103,"server-negotiator.queue_time_us":68}
I20260811 00:12:27.442734 10399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.427771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:45393 (local address 127.0.0.1:59852)
0811 00:12:27.428523 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:27.428546 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:27.428723 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:27.429052 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:27.429056 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:27.429932 (+ 876us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:27.430236 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:27.430244 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.431254 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.431262 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.432256 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:27.432268 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.433012 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.433019 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.433177 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.433934 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:27.433960 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:27.436213 (+ 2253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:27.440005 (+ 3792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:27.440015 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:27.440021 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:27.440468 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:27.440855 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:27.440858 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:27.440860 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:27.441031 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:27.441624 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:27.441630 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:27.441778 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 00:12:27.441953 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 00:12:27.442222 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":402,"threads_started":1}
I20260811 00:12:27.448339 9509 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:59852:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 00:12:27.449748 9509 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:12:27.456878 9745 tablet_service.cc:1467] Processing CreateTablet for tablet e42c56403476485e8d94caaac51929cb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=61f7ef5f60094efb8f9dac8a37da01f6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:12:27.457494 9745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42c56403476485e8d94caaac51929cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:27.457286 9963 tablet_service.cc:1467] Processing CreateTablet for tablet e42c56403476485e8d94caaac51929cb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=61f7ef5f60094efb8f9dac8a37da01f6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:12:27.457422 10188 tablet_service.cc:1467] Processing CreateTablet for tablet e42c56403476485e8d94caaac51929cb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=61f7ef5f60094efb8f9dac8a37da01f6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:12:27.457908 9963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42c56403476485e8d94caaac51929cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:27.457952 10188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42c56403476485e8d94caaac51929cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:27.465123 10344 tablet_bootstrap.cc:492] T e42c56403476485e8d94caaac51929cb P 6e7620e8b453422b9c5a06dd253d21f4: Bootstrap starting.
I20260811 00:12:27.465838 10343 tablet_bootstrap.cc:492] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38: Bootstrap starting.
I20260811 00:12:27.466459 10342 tablet_bootstrap.cc:492] T e42c56403476485e8d94caaac51929cb P d7d2e0a2066341a68280263ab5a6c013: Bootstrap starting.
I20260811 00:12:27.466895 10344 tablet_bootstrap.cc:654] T e42c56403476485e8d94caaac51929cb P 6e7620e8b453422b9c5a06dd253d21f4: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:27.467411 10343 tablet_bootstrap.cc:654] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:27.468233 10342 tablet_bootstrap.cc:654] T e42c56403476485e8d94caaac51929cb P d7d2e0a2066341a68280263ab5a6c013: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:27.469363 10344 tablet_bootstrap.cc:492] T e42c56403476485e8d94caaac51929cb P 6e7620e8b453422b9c5a06dd253d21f4: No bootstrap required, opened a new log
I20260811 00:12:27.469516 10344 ts_tablet_manager.cc:1397] T e42c56403476485e8d94caaac51929cb P 6e7620e8b453422b9c5a06dd253d21f4: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260811 00:12:27.470501 10344 raft_consensus.cc:348] T e42c56403476485e8d94caaac51929cb P 6e7620e8b453422b9c5a06dd253d21f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } }
I20260811 00:12:27.470749 10344 raft_consensus.cc:374] T e42c56403476485e8d94caaac51929cb P 6e7620e8b453422b9c5a06dd253d21f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:27.470773 10342 tablet_bootstrap.cc:492] T e42c56403476485e8d94caaac51929cb P d7d2e0a2066341a68280263ab5a6c013: No bootstrap required, opened a new log
I20260811 00:12:27.470834 10344 raft_consensus.cc:729] T e42c56403476485e8d94caaac51929cb P 6e7620e8b453422b9c5a06dd253d21f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e7620e8b453422b9c5a06dd253d21f4, State: Initialized, Role: FOLLOWER
I20260811 00:12:27.470944 10342 ts_tablet_manager.cc:1397] T e42c56403476485e8d94caaac51929cb P d7d2e0a2066341a68280263ab5a6c013: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260811 00:12:27.471062 10344 consensus_queue.cc:260] T e42c56403476485e8d94caaac51929cb P 6e7620e8b453422b9c5a06dd253d21f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } }
I20260811 00:12:27.471526 10351 tablet_replica.cc:439] T e42c56403476485e8d94caaac51929cb P 6e7620e8b453422b9c5a06dd253d21f4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } } }
I20260811 00:12:27.471709 10342 raft_consensus.cc:348] T e42c56403476485e8d94caaac51929cb P d7d2e0a2066341a68280263ab5a6c013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } }
I20260811 00:12:27.471855 10351 tablet_replica.cc:442] T e42c56403476485e8d94caaac51929cb P 6e7620e8b453422b9c5a06dd253d21f4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:12:27.471951 10342 raft_consensus.cc:374] T e42c56403476485e8d94caaac51929cb P d7d2e0a2066341a68280263ab5a6c013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:27.472029 10342 raft_consensus.cc:729] T e42c56403476485e8d94caaac51929cb P d7d2e0a2066341a68280263ab5a6c013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7d2e0a2066341a68280263ab5a6c013, State: Initialized, Role: FOLLOWER
I20260811 00:12:27.472366 10344 ts_tablet_manager.cc:1428] T e42c56403476485e8d94caaac51929cb P 6e7620e8b453422b9c5a06dd253d21f4: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 00:12:27.472360 10342 consensus_queue.cc:260] T e42c56403476485e8d94caaac51929cb P d7d2e0a2066341a68280263ab5a6c013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } }
I20260811 00:12:27.472803 10350 tablet_replica.cc:439] T e42c56403476485e8d94caaac51929cb P d7d2e0a2066341a68280263ab5a6c013: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } } }
I20260811 00:12:27.473150 10350 tablet_replica.cc:442] T e42c56403476485e8d94caaac51929cb P d7d2e0a2066341a68280263ab5a6c013: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:12:27.474066 10342 ts_tablet_manager.cc:1428] T e42c56403476485e8d94caaac51929cb P d7d2e0a2066341a68280263ab5a6c013: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 00:12:27.477304 10343 tablet_bootstrap.cc:492] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38: No bootstrap required, opened a new log
I20260811 00:12:27.477488 10343 ts_tablet_manager.cc:1397] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.004s
I20260811 00:12:27.478317 10343 raft_consensus.cc:348] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } }
I20260811 00:12:27.478567 10343 raft_consensus.cc:374] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:27.478680 10343 raft_consensus.cc:729] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd92062178ec4d9ca39fd0a82d996b38, State: Initialized, Role: FOLLOWER
I20260811 00:12:27.478919 10343 consensus_queue.cc:260] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } }
I20260811 00:12:27.479521 10343 ts_tablet_manager.cc:1428] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 00:12:27.479403 10369 tablet_replica.cc:439] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } } }
I20260811 00:12:27.479725 10369 tablet_replica.cc:442] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:12:27.598691 10369 raft_consensus.cc:482] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:12:27.598917 10369 raft_consensus.cc:504] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } }
I20260811 00:12:27.599673 10369 leader_election.cc:288] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e7620e8b453422b9c5a06dd253d21f4 (127.7.94.1:42743), d7d2e0a2066341a68280263ab5a6c013 (127.7.94.2:44173)
I20260811 00:12:27.600270 10005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42c56403476485e8d94caaac51929cb" candidate_uuid: "bd92062178ec4d9ca39fd0a82d996b38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7d2e0a2066341a68280263ab5a6c013" is_pre_election: true
I20260811 00:12:27.600296 9781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42c56403476485e8d94caaac51929cb" candidate_uuid: "bd92062178ec4d9ca39fd0a82d996b38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7620e8b453422b9c5a06dd253d21f4" is_pre_election: true
I20260811 00:12:27.600567 10005 raft_consensus.cc:2454] T e42c56403476485e8d94caaac51929cb P d7d2e0a2066341a68280263ab5a6c013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd92062178ec4d9ca39fd0a82d996b38 in term 0.
I20260811 00:12:27.600557 9781 raft_consensus.cc:2454] T e42c56403476485e8d94caaac51929cb P 6e7620e8b453422b9c5a06dd253d21f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd92062178ec4d9ca39fd0a82d996b38 in term 0.
I20260811 00:12:27.601009 10107 leader_election.cc:302] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd92062178ec4d9ca39fd0a82d996b38, d7d2e0a2066341a68280263ab5a6c013; no voters:
I20260811 00:12:27.601382 10369 raft_consensus.cc:2781] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:12:27.601539 10369 raft_consensus.cc:482] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:12:27.601652 10369 raft_consensus.cc:3037] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:27.604206 10369 raft_consensus.cc:504] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } }
I20260811 00:12:27.604838 10369 leader_election.cc:288] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38 [CANDIDATE]: Term 1 election: Requested vote from peers 6e7620e8b453422b9c5a06dd253d21f4 (127.7.94.1:42743), d7d2e0a2066341a68280263ab5a6c013 (127.7.94.2:44173)
I20260811 00:12:27.605279 9781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42c56403476485e8d94caaac51929cb" candidate_uuid: "bd92062178ec4d9ca39fd0a82d996b38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7620e8b453422b9c5a06dd253d21f4"
I20260811 00:12:27.605393 10005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42c56403476485e8d94caaac51929cb" candidate_uuid: "bd92062178ec4d9ca39fd0a82d996b38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7d2e0a2066341a68280263ab5a6c013"
I20260811 00:12:27.605511 9781 raft_consensus.cc:3037] T e42c56403476485e8d94caaac51929cb P 6e7620e8b453422b9c5a06dd253d21f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:27.605623 10005 raft_consensus.cc:3037] T e42c56403476485e8d94caaac51929cb P d7d2e0a2066341a68280263ab5a6c013 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:27.608479 10005 raft_consensus.cc:2454] T e42c56403476485e8d94caaac51929cb P d7d2e0a2066341a68280263ab5a6c013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd92062178ec4d9ca39fd0a82d996b38 in term 1.
I20260811 00:12:27.608494 9781 raft_consensus.cc:2454] T e42c56403476485e8d94caaac51929cb P 6e7620e8b453422b9c5a06dd253d21f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd92062178ec4d9ca39fd0a82d996b38 in term 1.
I20260811 00:12:27.608906 10108 leader_election.cc:302] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e7620e8b453422b9c5a06dd253d21f4, bd92062178ec4d9ca39fd0a82d996b38; no voters:
I20260811 00:12:27.609254 10369 raft_consensus.cc:2781] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:27.609412 10369 raft_consensus.cc:686] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38 [term 1 LEADER]: Becoming Leader. State: Replica: bd92062178ec4d9ca39fd0a82d996b38, State: Running, Role: LEADER
I20260811 00:12:27.609760 10369 consensus_queue.cc:237] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } }
I20260811 00:12:27.611539 10352 tablet_replica.cc:439] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38: TxnStatusTablet state changed. Reason: New leader bd92062178ec4d9ca39fd0a82d996b38. Latest consensus state: current_term: 1 leader_uuid: "bd92062178ec4d9ca39fd0a82d996b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } } }
I20260811 00:12:27.611891 10352 tablet_replica.cc:442] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:12:27.612874 10404 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 00:12:27.613232 10404 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 00:12:27.613395 9510 catalog_manager.cc:5495] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd92062178ec4d9ca39fd0a82d996b38 (127.7.94.3). New cstate: current_term: 1 leader_uuid: "bd92062178ec4d9ca39fd0a82d996b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } health_report { overall_health: HEALTHY } } }
I20260811 00:12:27.638381 10338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.632546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33412 (local address 127.7.94.3:37631)
0811 00:12:27.632731 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:27.632736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:27.632873 (+ 137us) server_negotiation.cc:407] Connection header received
0811 00:12:27.633193 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:27.633197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:27.633312 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:27.633384 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:12:27.633886 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.634648 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.636627 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.637065 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.637170 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.637747 (+ 577us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:12:27.637849 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:12:27.637916 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:27.638089 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 00:12:27.638160 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 00:12:27.638535 10399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.632487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:37631 (local address 127.0.0.1:33412)
0811 00:12:27.632778 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:27.632801 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:27.632990 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:27.633463 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:27.633468 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:27.633482 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:12:27.633744 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:27.633758 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.634797 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.634803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.636484 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:27.636501 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.637260 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.637274 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.637485 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.637572 (+ 87us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:12:27.637945 (+ 373us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:12:27.637967 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 00:12:27.638063 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 00:12:27.638221 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 00:12:27.642254 9781 raft_consensus.cc:1261] T e42c56403476485e8d94caaac51929cb P 6e7620e8b453422b9c5a06dd253d21f4 [term 1 FOLLOWER]: Refusing update from remote peer bd92062178ec4d9ca39fd0a82d996b38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:12:27.642402 10005 raft_consensus.cc:1261] T e42c56403476485e8d94caaac51929cb P d7d2e0a2066341a68280263ab5a6c013 [term 1 FOLLOWER]: Refusing update from remote peer bd92062178ec4d9ca39fd0a82d996b38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:12:27.643003 10352 consensus_queue.cc:1035] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:12:27.643549 10369 consensus_queue.cc:1035] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:12:27.647087 10350 tablet_replica.cc:439] T e42c56403476485e8d94caaac51929cb P d7d2e0a2066341a68280263ab5a6c013: TxnStatusTablet state changed. Reason: New leader bd92062178ec4d9ca39fd0a82d996b38. Latest consensus state: current_term: 1 leader_uuid: "bd92062178ec4d9ca39fd0a82d996b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } } }
I20260811 00:12:27.647404 10350 tablet_replica.cc:442] T e42c56403476485e8d94caaac51929cb P d7d2e0a2066341a68280263ab5a6c013: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:12:27.650151 10369 tablet_replica.cc:439] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bd92062178ec4d9ca39fd0a82d996b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } } }
I20260811 00:12:27.650414 10369 tablet_replica.cc:442] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:12:27.655310 10361 mvcc.cc:205] Tried to move back new op lower bound from 7317123676736589824 to 7317123676611846144. Current Snapshot: MvccSnapshot[applied={T|T < 7317123676736589824 or (T in {7317123676736589824})}]
I20260811 00:12:27.655735 10350 tablet_replica.cc:439] T e42c56403476485e8d94caaac51929cb P d7d2e0a2066341a68280263ab5a6c013: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bd92062178ec4d9ca39fd0a82d996b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } } }
I20260811 00:12:27.655963 10350 tablet_replica.cc:442] T e42c56403476485e8d94caaac51929cb P d7d2e0a2066341a68280263ab5a6c013: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:12:27.658777 10351 tablet_replica.cc:439] T e42c56403476485e8d94caaac51929cb P 6e7620e8b453422b9c5a06dd253d21f4: TxnStatusTablet state changed. Reason: New leader bd92062178ec4d9ca39fd0a82d996b38. Latest consensus state: current_term: 1 leader_uuid: "bd92062178ec4d9ca39fd0a82d996b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } } }
I20260811 00:12:27.659099 10351 tablet_replica.cc:442] T e42c56403476485e8d94caaac51929cb P 6e7620e8b453422b9c5a06dd253d21f4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:12:27.661521 10352 tablet_replica.cc:439] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bd92062178ec4d9ca39fd0a82d996b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } } }
I20260811 00:12:27.661859 10352 tablet_replica.cc:442] T e42c56403476485e8d94caaac51929cb P bd92062178ec4d9ca39fd0a82d996b38: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:12:27.663717 10351 tablet_replica.cc:439] T e42c56403476485e8d94caaac51929cb P 6e7620e8b453422b9c5a06dd253d21f4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bd92062178ec4d9ca39fd0a82d996b38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7620e8b453422b9c5a06dd253d21f4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42743 } } peers { permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 } } peers { permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 } } }
I20260811 00:12:27.664045 10351 tablet_replica.cc:442] T e42c56403476485e8d94caaac51929cb P 6e7620e8b453422b9c5a06dd253d21f4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:12:27.669272 10412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.659422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34294 (local address 127.7.94.1:42743)
0811 00:12:27.661049 (+ 1627us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:27.661055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:27.661083 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:12:27.661187 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:27.661196 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:27.661351 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:27.662383 (+ 1032us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:12:27.662924 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.665416 (+ 2492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.666850 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.667311 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.667456 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.667932 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:12:27.668570 (+ 638us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:12:27.668749 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:27.668811 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 00:12:27.668904 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1416,"thread_start_us":1284,"threads_started":1}
I20260811 00:12:27.685998 10338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.679916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:33421 (local address 127.7.94.3:37631)
0811 00:12:27.680086 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:27.680092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:27.680659 (+ 567us) server_negotiation.cc:407] Connection header received
0811 00:12:27.680809 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:27.680813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:27.680872 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:27.680940 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:12:27.681448 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.682190 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.684276 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.684749 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.684840 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.685392 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:12:27.685507 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:12:27.685574 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:27.685720 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 00:12:27.685778 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 00:12:27.686230 10420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.679867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:37631 (local address 127.7.94.1:33421)
0811 00:12:27.680558 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:27.680577 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:27.680704 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:27.680976 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:27.680981 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:27.680993 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:12:27.681305 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:27.681315 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.682323 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.682329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.684101 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:27.684117 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.684924 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.684931 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.685103 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.685192 (+ 89us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:12:27.685603 (+ 411us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:12:27.685614 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 00:12:27.685692 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 00:12:27.685827 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":400,"threads_started":1}
I20260811 00:12:27.696719 10005 raft_consensus.cc:1261] T 60962ad60de54b0f85a190f335c8234f P d7d2e0a2066341a68280263ab5a6c013 [term 1 FOLLOWER]: Refusing update from remote peer 6e7620e8b453422b9c5a06dd253d21f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:12:27.696826 10233 raft_consensus.cc:1261] T 60962ad60de54b0f85a190f335c8234f P bd92062178ec4d9ca39fd0a82d996b38 [term 1 FOLLOWER]: Refusing update from remote peer 6e7620e8b453422b9c5a06dd253d21f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:12:27.697497 10390 consensus_queue.cc:1035] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7d2e0a2066341a68280263ab5a6c013" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 44173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:12:27.697870 10351 consensus_queue.cc:1035] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd92062178ec4d9ca39fd0a82d996b38" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:12:27.702248 10362 mvcc.cc:205] Tried to move back new op lower bound from 7317123676958691328 to 7317123675673468928. Current Snapshot: MvccSnapshot[applied={T|T < 7317123676958691328}]
I20260811 00:12:27.735850 10412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.728716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:60527 (local address 127.7.94.1:42743)
0811 00:12:27.728913 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:27.728920 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:27.729777 (+ 857us) server_negotiation.cc:407] Connection header received
0811 00:12:27.730058 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:27.730064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:27.730158 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:27.730252 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:12:27.731039 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.732063 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.733798 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.734330 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.734429 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.735071 (+ 642us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:12:27.735187 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:12:27.735239 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:27.735446 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 00:12:27.735522 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:12:27.735980 10429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.728730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:42743 (local address 127.7.94.3:60527)
0811 00:12:27.729683 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:27.729704 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:27.729904 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:27.730291 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:27.730296 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:27.730308 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:12:27.730835 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:27.730860 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.732204 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.732210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.733621 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:27.733634 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.734628 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.734633 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:27.734798 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.734877 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:12:27.735333 (+ 456us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:12:27.735342 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 00:12:27.735421 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 00:12:27.735559 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":511,"threads_started":1}
I20260811 00:12:27.776021 9511 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:12:27.776295 9511 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:12:27.779809 9511 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a6450d6ee5e4752a771d83b158a97a9: Sending DeleteTablet for 3 replicas of tablet 60962ad60de54b0f85a190f335c8234f
I20260811 00:12:27.780479 9745 tablet_service.cc:1514] Processing DeleteTablet for tablet 60962ad60de54b0f85a190f335c8234f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:27 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:34290
I20260811 00:12:27.780884 9963 tablet_service.cc:1514] Processing DeleteTablet for tablet 60962ad60de54b0f85a190f335c8234f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:27 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:60852
I20260811 00:12:27.783010 10188 tablet_service.cc:1514] Processing DeleteTablet for tablet 60962ad60de54b0f85a190f335c8234f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:27 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:33402
I20260811 00:12:27.793617 10440 tablet_replica.cc:330] T 60962ad60de54b0f85a190f335c8234f P bd92062178ec4d9ca39fd0a82d996b38: stopping tablet replica
I20260811 00:12:27.793884 10440 raft_consensus.cc:2229] T 60962ad60de54b0f85a190f335c8234f P bd92062178ec4d9ca39fd0a82d996b38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:12:27.793962 10436 tablet_replica.cc:330] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4: stopping tablet replica
I20260811 00:12:27.794179 10440 raft_consensus.cc:2258] T 60962ad60de54b0f85a190f335c8234f P bd92062178ec4d9ca39fd0a82d996b38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:27.794643 10436 raft_consensus.cc:2229] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:12:27.795351 10440 ts_tablet_manager.cc:1905] T 60962ad60de54b0f85a190f335c8234f P bd92062178ec4d9ca39fd0a82d996b38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:27.795349 10436 raft_consensus.cc:2258] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:27.796679 10436 ts_tablet_manager.cc:1905] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:27.799882 10438 tablet_replica.cc:330] T 60962ad60de54b0f85a190f335c8234f P d7d2e0a2066341a68280263ab5a6c013: stopping tablet replica
I20260811 00:12:27.800118 10438 raft_consensus.cc:2229] T 60962ad60de54b0f85a190f335c8234f P d7d2e0a2066341a68280263ab5a6c013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:12:27.800421 10438 raft_consensus.cc:2258] T 60962ad60de54b0f85a190f335c8234f P d7d2e0a2066341a68280263ab5a6c013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:27.804850 10436 ts_tablet_manager.cc:1918] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:12:27.805059 10436 log.cc:1196] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/60962ad60de54b0f85a190f335c8234f
I20260811 00:12:27.805634 10438 ts_tablet_manager.cc:1905] T 60962ad60de54b0f85a190f335c8234f P d7d2e0a2066341a68280263ab5a6c013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:27.805970 10436 ts_tablet_manager.cc:1939] T 60962ad60de54b0f85a190f335c8234f P 6e7620e8b453422b9c5a06dd253d21f4: Deleting consensus metadata
I20260811 00:12:27.807332 9490 catalog_manager.cc:4841] TS 6e7620e8b453422b9c5a06dd253d21f4 (127.7.94.1:42743): tablet 60962ad60de54b0f85a190f335c8234f (table test-table [id=4035369e472842cc94911072bfe46508]) successfully deleted
I20260811 00:12:27.811048 10440 ts_tablet_manager.cc:1918] T 60962ad60de54b0f85a190f335c8234f P bd92062178ec4d9ca39fd0a82d996b38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:12:27.811278 10440 log.cc:1196] T 60962ad60de54b0f85a190f335c8234f P bd92062178ec4d9ca39fd0a82d996b38: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/60962ad60de54b0f85a190f335c8234f
I20260811 00:12:27.811853 10440 ts_tablet_manager.cc:1939] T 60962ad60de54b0f85a190f335c8234f P bd92062178ec4d9ca39fd0a82d996b38: Deleting consensus metadata
I20260811 00:12:27.812790 10438 ts_tablet_manager.cc:1918] T 60962ad60de54b0f85a190f335c8234f P d7d2e0a2066341a68280263ab5a6c013: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:12:27.812932 10438 log.cc:1196] T 60962ad60de54b0f85a190f335c8234f P d7d2e0a2066341a68280263ab5a6c013: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/60962ad60de54b0f85a190f335c8234f
I20260811 00:12:27.812978 9490 catalog_manager.cc:4841] TS bd92062178ec4d9ca39fd0a82d996b38 (127.7.94.3:37631): tablet 60962ad60de54b0f85a190f335c8234f (table test-table [id=4035369e472842cc94911072bfe46508]) successfully deleted
I20260811 00:12:27.814203 10438 ts_tablet_manager.cc:1939] T 60962ad60de54b0f85a190f335c8234f P d7d2e0a2066341a68280263ab5a6c013: Deleting consensus metadata
I20260811 00:12:27.815318 9489 catalog_manager.cc:4841] TS d7d2e0a2066341a68280263ab5a6c013 (127.7.94.2:44173): tablet 60962ad60de54b0f85a190f335c8234f (table test-table [id=4035369e472842cc94911072bfe46508]) successfully deleted
Aug 11 00:12:27 dist-test-slave-8hhm krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407146, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:12:27.831198 10412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.789520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34298 (local address 127.7.94.1:42743)
0811 00:12:27.789723 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:27.789732 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:27.801476 (+ 11744us) server_negotiation.cc:407] Connection header received
0811 00:12:27.801930 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:27.801937 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:27.802088 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:27.802208 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:27.813354 (+ 11146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.814583 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.816577 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.817274 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.817397 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.822826 (+ 5429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:27.822842 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:27.822861 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:27.822911 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:27.827299 (+ 4388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:27.828177 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:27.828187 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:27.828190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:27.828384 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:27.828802 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:27.828807 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:27.828811 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:27.829383 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:27.829774 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:27.830459 (+ 685us) server_negotiation.cc:299] Negotiation successful
0811 00:12:27.830770 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260811 00:12:27.832062 9846 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34298
I20260811 00:12:27.861290 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.842534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59864 (local address 127.7.94.62:45393)
0811 00:12:27.842813 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:27.842822 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:27.845752 (+ 2930us) server_negotiation.cc:407] Connection header received
0811 00:12:27.845937 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:27.845943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:27.846027 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:27.846128 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:27.847832 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.848957 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.850793 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.851425 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.851570 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.854716 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:27.854727 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:27.854733 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:27.854775 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:27.858248 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:27.859023 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:27.859028 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:27.859032 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:27.859174 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:27.859587 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:27.859592 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:27.859595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:27.859978 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:27.860248 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:27.860640 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 00:12:27.860873 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260811 00:12:27.861873 9511 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59864
I20260811 00:12:27.864054 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 9631
I20260811 00:12:27.889968 10412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.865369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34310 (local address 127.7.94.1:42743)
0811 00:12:27.865584 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:27.865591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:27.873691 (+ 8100us) server_negotiation.cc:407] Connection header received
0811 00:12:27.873855 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:27.873860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:27.873941 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:27.874038 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:27.875460 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.876505 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.878214 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.878956 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.879088 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.881895 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:27.881903 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:27.881907 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:27.881944 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:27.885973 (+ 4029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:27.886793 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:27.886798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:27.886802 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:27.886966 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:27.887399 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:27.887404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:27.887408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:27.887775 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:27.888038 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:27.889111 (+ 1073us) server_negotiation.cc:299] Negotiation successful
0811 00:12:27.889552 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 00:12:27.894598 9846 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34310
W20260811 00:12:27.895320 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 9631: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:12:27.895468 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 9631
I20260811 00:12:27.932668 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 9860
Aug 11 00:12:27 dist-test-slave-8hhm krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407146, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.94.2@KRBTEST.COM
I20260811 00:12:27.952283 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.933475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60856 (local address 127.7.94.2:44173)
0811 00:12:27.934449 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:27.934456 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:27.934477 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:12:27.934544 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:27.934558 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:27.934634 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:27.934829 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:27.936286 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.937408 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.939203 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.939839 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.939958 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.945112 (+ 5154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:27.945129 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:27.945147 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:27.945192 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:27.949027 (+ 3835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:27.949991 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:27.950000 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:27.950003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:27.950158 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:27.950588 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:27.950594 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:27.950598 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:27.951026 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:27.951288 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:27.951648 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 00:12:27.951919 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":803,"thread_start_us":609,"threads_started":1}
W20260811 00:12:27.952992 10070 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60856
W20260811 00:12:27.953685 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 9860: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:12:27.953867 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 9860
I20260811 00:12:27.983515 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 10084
Aug 11 00:12:27 dist-test-slave-8hhm krb5kdc[9440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407146, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.94.3@KRBTEST.COM
I20260811 00:12:28.000147 10338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:27.984366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33424 (local address 127.7.94.3:37631)
0811 00:12:27.984591 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:27.984597 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:27.984623 (+ 26us) server_negotiation.cc:407] Connection header received
0811 00:12:27.984804 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:27.984810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:27.984905 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:27.985026 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:27.986390 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.987118 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.988249 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:27.988667 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:27.988801 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:27.993110 (+ 4309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:27.993130 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:27.993151 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:27.993202 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:27.997332 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:27.997943 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:27.997952 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:27.997957 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:27.998128 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:27.998464 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:27.998470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:27.998476 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:27.998961 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:27.999300 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:27.999480 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 00:12:27.999740 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 00:12:28.000942 10303 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33424
W20260811 00:12:28.001704 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 10084: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:12:28.001854 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 10084
I20260811 00:12:28.034662 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 9461
I20260811 00:12:28.035840 9577 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:28.151126 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 9461
2026-08-11T00:12:28Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4869 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:12:28 dist-test-slave-8hhm krb5kdc[10454](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:12:28 dist-test-slave-8hhm krb5kdc[10454](info): set up 2 sockets
Aug 11 00:12:28 dist-test-slave-8hhm krb5kdc[10454](info): commencing operation
krb5kdc: starting...
W20260811 00:12:29.544749 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.341s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:12:29 dist-test-slave-8hhm krb5kdc[10454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:12:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:12:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:12:29.738950 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:33893
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:44419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:33893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:30.063936 10473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:30.064282 10473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:30.064437 10473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:30.071468 10473 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:12:30.071601 10473 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 00:12:30.071705 10473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:30.071820 10473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:30.071906 10473 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:12:30.072003 10473 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:12:30.081549 10473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:44419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:33893
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:33893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:30.082783 10473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:30.084100 10473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:30.095849 10483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:30.095803 10486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:30.095875 10484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:30.097010 10473 server_base.cc:1034] running on GCE node
I20260811 00:12:30.097788 10473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:30.099051 10473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:30.100353 10473 hybrid_clock.cc:648] HybridClock initialized: now 1786407150100285 us; error 46 us; skew 500 ppm
I20260811 00:12:30.100734 10473 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:30 dist-test-slave-8hhm krb5kdc[10454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407150, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:30.104178 10473 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:12:30.105871 10473 webserver.cc:460] Webserver started at http://127.7.94.62:44783/ using document root <none> and password file <none>
I20260811 00:12:30.106604 10473 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:30.106750 10473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:30.107097 10473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:30.110329 10473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "a158352fd6fb49d78036e80cc18fcbdc"
format_stamp: "Formatted at 2026-08-11 00:12:30 on dist-test-slave-8hhm"
server_key: "9302f9f25e6805c3c9f52a29a03ee9cf"
server_key_iv: "580897e15230409043318153fa50241e"
server_key_version: "encryptionkey@0"
I20260811 00:12:30.111236 10473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "a158352fd6fb49d78036e80cc18fcbdc"
format_stamp: "Formatted at 2026-08-11 00:12:30 on dist-test-slave-8hhm"
server_key: "9302f9f25e6805c3c9f52a29a03ee9cf"
server_key_iv: "580897e15230409043318153fa50241e"
server_key_version: "encryptionkey@0"
I20260811 00:12:30.116755 10473 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260811 00:12:30.120569 10497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:30.121853 10473 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:12:30.122046 10473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "a158352fd6fb49d78036e80cc18fcbdc"
format_stamp: "Formatted at 2026-08-11 00:12:30 on dist-test-slave-8hhm"
server_key: "9302f9f25e6805c3c9f52a29a03ee9cf"
server_key_iv: "580897e15230409043318153fa50241e"
server_key_version: "encryptionkey@0"
I20260811 00:12:30.122277 10473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:30.145514 10473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:30.149657 10473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:30.149950 10473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:30.171519 10473 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:33893
I20260811 00:12:30.171527 10594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:33893 every 8 connection(s)
I20260811 00:12:30.172881 10473 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:12:30.176218 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 10473
I20260811 00:12:30.176429 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:12:30.176915 10596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:30.177309 7544 external_mini_cluster.cc:1428] Setting key b928d3d874422fe9e3df00038a14c3e5
I20260811 00:12:30.191862 10596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc: Bootstrap starting.
Aug 11 00:12:30 dist-test-slave-8hhm krb5kdc[10454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:30.195602 10596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:30.196903 10596 log.cc:824] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:30.200138 10596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc: No bootstrap required, opened a new log
I20260811 00:12:30.201700 10603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:30.179453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55158 (local address 127.7.94.62:33893)
0811 00:12:30.180283 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:30.180294 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:30.180334 (+ 40us) server_negotiation.cc:407] Connection header received
0811 00:12:30.181995 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:30.182017 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:30.182417 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:30.182740 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:30.184526 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:30.185868 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:30.187729 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:30.188508 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:30.188643 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:30.193337 (+ 4694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:30.193349 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:30.193374 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:30.193422 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:30.197317 (+ 3895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:30.198166 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:30.198171 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:30.198176 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:30.198400 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:30.198754 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:30.198759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:30.198762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:30.199402 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:30.199709 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:30.199970 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 00:12:30.200356 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":420,"threads_started":1}
I20260811 00:12:30.208420 10596 raft_consensus.cc:348] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a158352fd6fb49d78036e80cc18fcbdc" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 33893 } }
I20260811 00:12:30.208789 10596 raft_consensus.cc:374] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:30.208878 10596 raft_consensus.cc:729] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a158352fd6fb49d78036e80cc18fcbdc, State: Initialized, Role: FOLLOWER
I20260811 00:12:30.209450 10596 consensus_queue.cc:260] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a158352fd6fb49d78036e80cc18fcbdc" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 33893 } }
I20260811 00:12:30.209647 10596 raft_consensus.cc:388] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:12:30.209745 10596 raft_consensus.cc:482] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:12:30.209860 10596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:30.212075 10596 raft_consensus.cc:504] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a158352fd6fb49d78036e80cc18fcbdc" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 33893 } }
I20260811 00:12:30.212494 10596 leader_election.cc:302] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a158352fd6fb49d78036e80cc18fcbdc; no voters:
I20260811 00:12:30.212968 10596 leader_election.cc:288] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:12:30.213102 10606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:30.213855 10606 raft_consensus.cc:686] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc [term 1 LEADER]: Becoming Leader. State: Replica: a158352fd6fb49d78036e80cc18fcbdc, State: Running, Role: LEADER
I20260811 00:12:30.214365 10606 consensus_queue.cc:237] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a158352fd6fb49d78036e80cc18fcbdc" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 33893 } }
I20260811 00:12:30.214990 10596 sys_catalog.cc:564] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:12:30.217046 10607 sys_catalog.cc:455] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a158352fd6fb49d78036e80cc18fcbdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a158352fd6fb49d78036e80cc18fcbdc" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 33893 } } }
I20260811 00:12:30.217509 10607 sys_catalog.cc:458] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:30.218375 10608 sys_catalog.cc:455] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a158352fd6fb49d78036e80cc18fcbdc. Latest consensus state: current_term: 1 leader_uuid: "a158352fd6fb49d78036e80cc18fcbdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a158352fd6fb49d78036e80cc18fcbdc" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 33893 } } }
I20260811 00:12:30.218776 10608 sys_catalog.cc:458] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:30.219147 10620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:12:30.225011 10620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:12:30.234632 10620 catalog_manager.cc:1347] Generated new cluster ID: 6676e63ebe7a4b52b11c2262bd67a959
I20260811 00:12:30.234745 10620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:12:30.244459 10620 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:12:30.245961 10620 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:12:30.253741 10620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc: Generated new TSK 0
I20260811 00:12:30.254456 10620 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
I20260811 00:12:30.304337 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:33893
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:44419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:30.630116 10641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:30.630383 10641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:30.630548 10641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:30.637629 10641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:30.637758 10641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:30.637920 10641 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 00:12:30.638007 10641 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:12:30.638114 10641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:12:30.647536 10641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:44419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.94.62:33893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:30.648772 10641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:30.650130 10641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:30.663036 10651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:30.663318 10647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:30.663687 10653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:30.663964 10641 server_base.cc:1034] running on GCE node
I20260811 00:12:30.664621 10641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:30.665558 10641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:30.666749 10641 hybrid_clock.cc:648] HybridClock initialized: now 1786407150666693 us; error 50 us; skew 500 ppm
I20260811 00:12:30.667075 10641 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:30 dist-test-slave-8hhm krb5kdc[10454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407150, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:30.670239 10641 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:12:30.671603 10641 webserver.cc:460] Webserver started at http://127.7.94.1:37151/ using document root <none> and password file <none>
I20260811 00:12:30.672279 10641 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:30.672422 10641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:30.672737 10641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:30.675611 10641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "18642798f024415cba62deab3184f14b"
format_stamp: "Formatted at 2026-08-11 00:12:30 on dist-test-slave-8hhm"
server_key: "8099580330d53d407322d025d016e669"
server_key_iv: "9b6de91f91e678acaef42bb87ec74945"
server_key_version: "encryptionkey@0"
I20260811 00:12:30.676370 10641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "18642798f024415cba62deab3184f14b"
format_stamp: "Formatted at 2026-08-11 00:12:30 on dist-test-slave-8hhm"
server_key: "8099580330d53d407322d025d016e669"
server_key_iv: "9b6de91f91e678acaef42bb87ec74945"
server_key_version: "encryptionkey@0"
I20260811 00:12:30.681483 10641 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260811 00:12:30.684702 10663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:30.685938 10641 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 00:12:30.686124 10641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "18642798f024415cba62deab3184f14b"
format_stamp: "Formatted at 2026-08-11 00:12:30 on dist-test-slave-8hhm"
server_key: "8099580330d53d407322d025d016e669"
server_key_iv: "9b6de91f91e678acaef42bb87ec74945"
server_key_version: "encryptionkey@0"
I20260811 00:12:30.686312 10641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:30.702875 10641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:30.707048 10641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:30.707322 10641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:30.709885 10641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:30.710031 10641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:30.710161 10641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:30.710397 10641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:12:30 dist-test-slave-8hhm krb5kdc[10454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407150, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:30.732647 10678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:30.710573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:44051 (local address 127.7.94.62:33893)
0811 00:12:30.711049 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:30.711058 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:30.712374 (+ 1316us) server_negotiation.cc:407] Connection header received
0811 00:12:30.714404 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:30.714409 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:30.714497 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:30.714610 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:30.716708 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:30.717781 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:30.719982 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:30.720577 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:30.720718 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:30.725200 (+ 4482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:30.725206 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:30.725251 (+ 45us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:30.725289 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:30.728776 (+ 3487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:30.729726 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:30.729733 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:30.729739 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:30.729878 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:30.730504 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:30.730513 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:30.730520 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:30.730989 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:30.731190 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:30.732033 (+ 843us) server_negotiation.cc:299] Negotiation successful
0811 00:12:30.732280 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":239,"threads_started":1}
I20260811 00:12:30.733840 10683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:30.711311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:33893 (local address 127.7.94.1:44051)
0811 00:12:30.712223 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:30.712268 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:30.714138 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:30.714942 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:30.714949 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:30.715685 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:30.716538 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:30.716557 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:30.717957 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:30.717966 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:30.719835 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:30.719847 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:30.720815 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:30.720822 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:30.721168 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:30.721983 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:30.722021 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:30.724987 (+ 2966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:30.728925 (+ 3938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:30.728934 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:30.728947 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:30.729590 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:30.729997 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:30.730002 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:30.730006 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:30.730310 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:30.731132 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:30.731138 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:30.731627 (+ 489us) client_negotiation.cc:769] Sending connection context
0811 00:12:30.731925 (+ 298us) client_negotiation.cc:240] Negotiation successful
0811 00:12:30.732421 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":370,"threads_started":1}
I20260811 00:12:30.761198 10678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:30.741711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:58005 (local address 127.7.94.62:33893)
0811 00:12:30.741930 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:30.741937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:30.742579 (+ 642us) server_negotiation.cc:407] Connection header received
0811 00:12:30.742800 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:30.742809 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:30.742909 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:30.743026 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:30.743915 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:30.745309 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:30.748412 (+ 3103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:30.749379 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:30.749560 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:30.752627 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:30.752635 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:30.752642 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:30.752691 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:30.757231 (+ 4540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:30.758331 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:30.758336 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:30.758340 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:30.758530 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:30.759075 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:30.759082 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:30.759085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:30.759532 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:30.759710 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:30.760612 (+ 902us) server_negotiation.cc:299] Negotiation successful
0811 00:12:30.760835 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 00:12:30.761333 10787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:30.741723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:33893 (local address 127.7.94.1:58005)
0811 00:12:30.742450 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:30.742480 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:30.742646 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:30.743083 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:30.743087 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:30.743491 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:30.743772 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:30.743780 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:30.745553 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:30.745560 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:30.748162 (+ 2602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:30.748188 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:30.749876 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:30.749884 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:30.750082 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:30.750748 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:30.750782 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:30.752283 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:30.757486 (+ 5203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:30.757506 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:30.757513 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:30.758173 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:30.758700 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:30.758708 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:30.758715 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:30.758909 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:30.759734 (+ 825us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:30.759742 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:30.760188 (+ 446us) client_negotiation.cc:769] Sending connection context
0811 00:12:30.760485 (+ 297us) client_negotiation.cc:240] Negotiation successful
0811 00:12:30.760825 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":395,"threads_started":1}
I20260811 00:12:30.767935 10641 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:37555
I20260811 00:12:30.768028 10870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:37555 every 8 connection(s)
I20260811 00:12:30.769438 10641 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:12:30.771423 10871 heartbeater.cc:347] Connected to a master server at 127.7.94.62:33893
I20260811 00:12:30.771652 10871 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:30.772322 10871 heartbeater.cc:511] Master 127.7.94.62:33893 requested a full tablet report, sending...
I20260811 00:12:30.773391 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 10641
I20260811 00:12:30.773626 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:12:30.774441 7544 external_mini_cluster.cc:1428] Setting key aab372291aff176a5908fa0ffa3ccc43
I20260811 00:12:30.774566 10529 ts_manager.cc:194] Registered new tserver with Master: 18642798f024415cba62deab3184f14b (127.7.94.1:37555)
I20260811 00:12:30.776691 10529 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:44051
WARNING: no policy specified for kudu/127.7.94.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
I20260811 00:12:30.822715 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:33893
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:44419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:31.164831 10877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:31.165141 10877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:31.165349 10877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:31.172547 10877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:31.172653 10877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:31.172837 10877 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 00:12:31.172925 10877 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:12:31.173072 10877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:12:31.182998 10877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:44419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.94.62:33893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:31.184298 10877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:31.185781 10877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:31.198546 10888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:31.198629 10889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:31.198572 10891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:31.202569 10877 server_base.cc:1034] running on GCE node
I20260811 00:12:31.203258 10877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:31.204135 10877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:31.205312 10877 hybrid_clock.cc:648] HybridClock initialized: now 1786407151205263 us; error 31 us; skew 500 ppm
I20260811 00:12:31.205613 10877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:31 dist-test-slave-8hhm krb5kdc[10454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407151, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:31.213465 10877 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:12:31.214913 10877 webserver.cc:460] Webserver started at http://127.7.94.2:34711/ using document root <none> and password file <none>
I20260811 00:12:31.215638 10877 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:31.215778 10877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:31.216058 10877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:31.218926 10877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "bd9b2df5cede4dc19d32067269d7e50c"
format_stamp: "Formatted at 2026-08-11 00:12:31 on dist-test-slave-8hhm"
server_key: "819ab5230866a94464e9ea16b8a0c22e"
server_key_iv: "249b47e944ab2356aff10091c0a46cca"
server_key_version: "encryptionkey@0"
I20260811 00:12:31.219628 10877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "bd9b2df5cede4dc19d32067269d7e50c"
format_stamp: "Formatted at 2026-08-11 00:12:31 on dist-test-slave-8hhm"
server_key: "819ab5230866a94464e9ea16b8a0c22e"
server_key_iv: "249b47e944ab2356aff10091c0a46cca"
server_key_version: "encryptionkey@0"
I20260811 00:12:31.224555 10877 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 00:12:31.228021 10901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:31.229358 10877 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:12:31.229516 10877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "bd9b2df5cede4dc19d32067269d7e50c"
format_stamp: "Formatted at 2026-08-11 00:12:31 on dist-test-slave-8hhm"
server_key: "819ab5230866a94464e9ea16b8a0c22e"
server_key_iv: "249b47e944ab2356aff10091c0a46cca"
server_key_version: "encryptionkey@0"
I20260811 00:12:31.229713 10877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:31.260571 10877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:31.264810 10877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:31.265100 10877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:31.267797 10877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:31.267927 10877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:31.268049 10877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:31.268141 10877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:12:31 dist-test-slave-8hhm krb5kdc[10454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407151, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:31.292510 10914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:31.268531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:55943 (local address 127.7.94.62:33893)
0811 00:12:31.269098 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:31.269104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:31.270252 (+ 1148us) server_negotiation.cc:407] Connection header received
0811 00:12:31.272423 (+ 2171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:31.272429 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:31.272514 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:31.272647 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:31.274774 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.275916 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.279048 (+ 3132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.279464 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.279560 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:31.285028 (+ 5468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:31.285037 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:31.285045 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:31.285097 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:31.288765 (+ 3668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:31.289894 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:31.289901 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:31.289907 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:31.290074 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:31.290661 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:31.290669 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:31.290671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:31.290952 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:31.291109 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:31.291878 (+ 769us) server_negotiation.cc:299] Negotiation successful
0811 00:12:31.292138 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":338,"threads_started":1}
I20260811 00:12:31.293664 10915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:31.269173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:33893 (local address 127.7.94.2:55943)
0811 00:12:31.270061 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:31.270125 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:31.272149 (+ 2024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:31.272939 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:31.272953 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:31.273705 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:31.274592 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:31.274625 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.276091 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.276100 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:31.278877 (+ 2777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:31.278898 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.279718 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.279728 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:31.280138 (+ 410us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:31.280930 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:31.280968 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:31.284751 (+ 3783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:31.288903 (+ 4152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:31.288910 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:31.288928 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:31.289731 (+ 803us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:31.290195 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:31.290203 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:31.290206 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:31.290539 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:31.291090 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:31.291105 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:31.291486 (+ 381us) client_negotiation.cc:769] Sending connection context
0811 00:12:31.291749 (+ 263us) client_negotiation.cc:240] Negotiation successful
0811 00:12:31.292249 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":358,"threads_started":1}
I20260811 00:12:31.314750 10914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:31.299400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:46335 (local address 127.7.94.62:33893)
0811 00:12:31.299549 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:31.299554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:31.300371 (+ 817us) server_negotiation.cc:407] Connection header received
0811 00:12:31.300555 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:31.300560 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:31.300633 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:31.300727 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:31.301906 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.302876 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.305006 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.305623 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.305727 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:31.308164 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:31.308168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:31.308172 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:31.308200 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:31.311609 (+ 3409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:31.312514 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:31.312521 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:31.312527 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:31.312701 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:31.313166 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:31.313172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:31.313177 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:31.313611 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:31.313764 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:31.314168 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 00:12:31.314390 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 00:12:31.314927 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:31.299543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:33893 (local address 127.7.94.2:46335)
0811 00:12:31.300265 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:31.300284 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:31.300449 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:31.300760 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:31.300764 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:31.301211 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:31.301695 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:31.301706 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.303071 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.303074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:31.304796 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:31.304818 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.306032 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.306041 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:31.306262 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:31.306928 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:31.306945 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:31.307919 (+ 974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:31.311745 (+ 3826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:31.311750 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:31.311754 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:31.312319 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:31.312845 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:31.312849 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:31.312851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:31.313021 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:31.313732 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:31.313737 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:31.313922 (+ 185us) client_negotiation.cc:769] Sending connection context
0811 00:12:31.314059 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 00:12:31.314445 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":360,"threads_started":1}
I20260811 00:12:31.318315 10877 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:34595
I20260811 00:12:31.318361 11120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:34595 every 8 connection(s)
I20260811 00:12:31.319770 10877 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:12:31.321638 11121 heartbeater.cc:347] Connected to a master server at 127.7.94.62:33893
I20260811 00:12:31.321820 11121 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:31.322407 11121 heartbeater.cc:511] Master 127.7.94.62:33893 requested a full tablet report, sending...
I20260811 00:12:31.323542 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 10877
I20260811 00:12:31.323729 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:12:31.323702 10529 ts_manager.cc:194] Registered new tserver with Master: bd9b2df5cede4dc19d32067269d7e50c (127.7.94.2:34595)
I20260811 00:12:31.324311 7544 external_mini_cluster.cc:1428] Setting key abb09f09224c836e4ec3c03c928ae804
I20260811 00:12:31.324754 10529 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:55943
WARNING: no policy specified for kudu/127.7.94.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
I20260811 00:12:31.369374 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:33893
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:44419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:31.697022 11125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:31.697367 11125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:31.697597 11125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:31.704494 11125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:31.704587 11125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:31.704666 11125 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 00:12:31.704737 11125 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:12:31.704804 11125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:12:31.713950 11125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:44419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.94.62:33893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:31.715274 11125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:31.716681 11125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:31.729776 11135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:31.729809 11131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:31.730275 11130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:31.730557 11125 server_base.cc:1034] running on GCE node
I20260811 00:12:31.731159 11125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:31.732064 11125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:31.733304 11125 hybrid_clock.cc:648] HybridClock initialized: now 1786407151733254 us; error 41 us; skew 500 ppm
I20260811 00:12:31.733603 11125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:31 dist-test-slave-8hhm krb5kdc[10454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407151, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:31.736800 11125 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:12:31.738226 11125 webserver.cc:460] Webserver started at http://127.7.94.3:38875/ using document root <none> and password file <none>
I20260811 00:12:31.738855 11125 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:31.738958 11125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:31.739223 11125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:31.742069 11125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "89b899be3f3b4dec819fdc0e2230e35d"
format_stamp: "Formatted at 2026-08-11 00:12:31 on dist-test-slave-8hhm"
server_key: "c840dfb2d665b2d7e05ec1597b41182b"
server_key_iv: "a0f54b9c5d2176151f1fabb5d3ea250c"
server_key_version: "encryptionkey@0"
I20260811 00:12:31.742791 11125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "89b899be3f3b4dec819fdc0e2230e35d"
format_stamp: "Formatted at 2026-08-11 00:12:31 on dist-test-slave-8hhm"
server_key: "c840dfb2d665b2d7e05ec1597b41182b"
server_key_iv: "a0f54b9c5d2176151f1fabb5d3ea250c"
server_key_version: "encryptionkey@0"
I20260811 00:12:31.747907 11125 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 00:12:31.751250 11144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:31.752518 11125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:12:31.752687 11125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "89b899be3f3b4dec819fdc0e2230e35d"
format_stamp: "Formatted at 2026-08-11 00:12:31 on dist-test-slave-8hhm"
server_key: "c840dfb2d665b2d7e05ec1597b41182b"
server_key_iv: "a0f54b9c5d2176151f1fabb5d3ea250c"
server_key_version: "encryptionkey@0"
I20260811 00:12:31.752902 11125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:31.768500 11125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:31.773679 11125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:31.773960 11125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:31.776486 11125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:31.776597 11125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:31.776713 11125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:31.776810 11125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:31.779596 10871 heartbeater.cc:503] Master 127.7.94.62:33893 was elected leader, sending a full tablet report...
Aug 11 00:12:31 dist-test-slave-8hhm krb5kdc[10454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407151, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:31.800606 10914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:31.777197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:60189 (local address 127.7.94.62:33893)
0811 00:12:31.777417 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:31.777423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:31.778748 (+ 1325us) server_negotiation.cc:407] Connection header received
0811 00:12:31.781313 (+ 2565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:31.781318 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:31.781402 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:31.781481 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:31.783905 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.784942 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.787595 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.788059 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.788202 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:31.793125 (+ 4923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:31.793130 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:31.793133 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:31.793170 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:31.796583 (+ 3413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:31.797627 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:31.797633 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:31.797637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:31.797826 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:31.798469 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:31.798472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:31.798474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:31.798845 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:31.798994 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:31.799995 (+ 1001us) server_negotiation.cc:299] Negotiation successful
0811 00:12:31.800246 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 00:12:31.802073 11162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:31.777650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:33893 (local address 127.7.94.3:60189)
0811 00:12:31.778556 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:31.778608 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:31.780923 (+ 2315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:31.782008 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:31.782023 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:31.782814 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:31.783701 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:31.783751 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.785143 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.785146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:31.787390 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:31.787410 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.788306 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.788314 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:31.788580 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:31.789534 (+ 954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:31.789560 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:31.792895 (+ 3335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:31.796725 (+ 3830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:31.796734 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:31.796749 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:31.797495 (+ 746us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:31.797971 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:31.797982 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:31.797989 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:31.798309 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:31.799066 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:31.799071 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:31.799560 (+ 489us) client_negotiation.cc:769] Sending connection context
0811 00:12:31.799893 (+ 333us) client_negotiation.cc:240] Negotiation successful
0811 00:12:31.800449 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":393,"threads_started":1}
I20260811 00:12:31.818650 11125 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:44257
I20260811 00:12:31.818760 11346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:44257 every 8 connection(s)
I20260811 00:12:31.820029 11125 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:12:31.822155 11347 heartbeater.cc:347] Connected to a master server at 127.7.94.62:33893
I20260811 00:12:31.822383 11347 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:31.823124 11347 heartbeater.cc:511] Master 127.7.94.62:33893 requested a full tablet report, sending...
I20260811 00:12:31.824317 11297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:31.809262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:33893 (local address 127.7.94.3:41457)
0811 00:12:31.810096 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:31.810121 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:31.810303 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:31.810627 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:31.810633 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:31.811208 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:31.811772 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:31.811784 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.813072 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.813079 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:31.814900 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:31.814916 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.816140 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.816159 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:31.816367 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:31.817022 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:31.817038 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:31.818072 (+ 1034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:31.821746 (+ 3674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:31.821751 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:31.821755 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:31.822177 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:31.822590 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:31.822596 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:31.822600 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:31.822772 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:31.823269 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:31.823273 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:31.823473 (+ 200us) client_negotiation.cc:769] Sending connection context
0811 00:12:31.823629 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 00:12:31.823989 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":450,"threads_started":1}
I20260811 00:12:31.824318 10914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:31.809264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:41457 (local address 127.7.94.62:33893)
0811 00:12:31.809431 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:31.809438 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:31.810221 (+ 783us) server_negotiation.cc:407] Connection header received
0811 00:12:31.810436 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:31.810439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:31.810494 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:31.810598 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:31.811926 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.812942 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.815065 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.815757 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.815879 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:31.818267 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:31.818272 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:31.818277 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:31.818311 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:31.821563 (+ 3252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:31.822310 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:31.822316 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:31.822319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:31.822488 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:31.822885 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:31.822889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:31.822891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:31.823145 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:31.823299 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:31.823720 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 00:12:31.823936 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 00:12:31.824708 10529 ts_manager.cc:194] Registered new tserver with Master: 89b899be3f3b4dec819fdc0e2230e35d (127.7.94.3:44257)
I20260811 00:12:31.825930 10529 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:60189
I20260811 00:12:31.827782 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 11125
I20260811 00:12:31.827963 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:12:31.828513 7544 external_mini_cluster.cc:1428] Setting key e26af598fc4f98fdca74eb73516b3201
I20260811 00:12:31.831187 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 00:12:31 dist-test-slave-8hhm krb5kdc[10454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 00:12:31 dist-test-slave-8hhm krb5kdc[10454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:31.884387 10914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:31.870944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48782 (local address 127.7.94.62:33893)
0811 00:12:31.871173 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:31.871180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:31.871382 (+ 202us) server_negotiation.cc:407] Connection header received
0811 00:12:31.871546 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:31.871551 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:31.871615 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:31.871712 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:31.873147 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.874034 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.875248 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.875650 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.875750 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:31.879501 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:31.879519 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:31.879523 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:31.879556 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:31.882135 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:31.882714 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:31.882718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:31.882720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:31.882861 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:31.883183 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:31.883186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:31.883188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:31.883440 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:31.883637 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:31.883846 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 00:12:31.884017 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:12:31.888512 10529 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:12:31.896005 10529 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:12:31.918581 11371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:31.908278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:34595 (local address 127.0.0.1:48608)
0811 00:12:31.909486 (+ 1208us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:31.909517 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:31.909716 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:31.910422 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:31.910430 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:31.910459 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:31.910815 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:31.910825 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.912572 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.912579 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:31.915761 (+ 3182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:31.915773 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.917704 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.917713 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:31.917894 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:31.917942 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 00:12:31.918022 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 00:12:31.918154 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1014,"mutex_wait_us":19,"thread_start_us":857,"threads_started":1}
I20260811 00:12:31.919976 11374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:31.908528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48608 (local address 127.7.94.2:34595)
0811 00:12:31.909084 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:31.909095 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:31.909639 (+ 544us) server_negotiation.cc:407] Connection header received
0811 00:12:31.909920 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:31.909931 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:31.910210 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:31.910409 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:31.910967 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.912420 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.915960 (+ 3540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.917456 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.919219 (+ 1763us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:31.919430 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:31.919496 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 00:12:31.919665 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":252,"threads_started":1}
I20260811 00:12:31.921200 11376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:31.908578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:37555 (local address 127.0.0.1:36346)
0811 00:12:31.911799 (+ 3221us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:31.911816 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:31.911939 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:31.912754 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:31.912761 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:31.912792 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:31.913337 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:31.913349 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.914552 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.914559 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:31.917632 (+ 3073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:31.917645 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.920249 (+ 2604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.920259 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:31.920554 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:31.920606 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 00:12:31.920712 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 00:12:31.920831 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3049,"mutex_wait_us":24,"thread_start_us":2922,"threads_started":1}
I20260811 00:12:31.921304 11378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:31.908812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36346 (local address 127.7.94.1:37555)
0811 00:12:31.910278 (+ 1466us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:31.910286 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:31.911918 (+ 1632us) server_negotiation.cc:407] Connection header received
0811 00:12:31.912088 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:31.912104 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:31.912310 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:31.912504 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:31.913504 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.914428 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.917811 (+ 3383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.919698 (+ 1887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.919859 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:31.920087 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:31.920791 (+ 704us) server_negotiation.cc:299] Negotiation successful
0811 00:12:31.920978 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1316,"thread_start_us":1166,"threads_started":1}
I20260811 00:12:31.924986 10991 tablet_service.cc:1467] Processing CreateTablet for tablet 159f7eed403c444194251aab51e55582 (DEFAULT_TABLE table=test-table [id=0068d79ecdd84ec5bb3695dea857c577]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:31.925028 10751 tablet_service.cc:1467] Processing CreateTablet for tablet 159f7eed403c444194251aab51e55582 (DEFAULT_TABLE table=test-table [id=0068d79ecdd84ec5bb3695dea857c577]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:31.926534 10991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 159f7eed403c444194251aab51e55582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:31.926534 10751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 159f7eed403c444194251aab51e55582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:31.929651 11371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:31.910375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:44257 (local address 127.0.0.1:44082)
0811 00:12:31.918940 (+ 8565us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:31.918957 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:31.919061 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:31.920139 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:31.920144 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:31.920176 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:31.920364 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:31.920380 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.922203 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.922208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:31.927205 (+ 4997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:31.927222 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.928849 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.928854 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:31.929039 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:31.929074 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 00:12:31.929165 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 00:12:31.929338 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8468,"thread_start_us":10351,"threads_started":1}
I20260811 00:12:31.929723 11380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:31.910752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44082 (local address 127.7.94.3:44257)
0811 00:12:31.911559 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:31.911567 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:31.919313 (+ 7746us) server_negotiation.cc:407] Connection header received
0811 00:12:31.919419 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:31.919427 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:31.919655 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:31.919834 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:31.920564 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.922007 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.927415 (+ 5408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.928675 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.928837 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:31.929073 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:31.929228 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 00:12:31.929406 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":445,"threads_started":1}
I20260811 00:12:31.932763 11228 tablet_service.cc:1467] Processing CreateTablet for tablet 159f7eed403c444194251aab51e55582 (DEFAULT_TABLE table=test-table [id=0068d79ecdd84ec5bb3695dea857c577]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:31.934422 11228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 159f7eed403c444194251aab51e55582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:31.942134 11382 tablet_bootstrap.cc:492] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c: Bootstrap starting.
I20260811 00:12:31.945802 11382 tablet_bootstrap.cc:654] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:31.945931 11381 tablet_bootstrap.cc:492] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b: Bootstrap starting.
I20260811 00:12:31.947244 11382 log.cc:824] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:31.947852 11383 tablet_bootstrap.cc:492] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d: Bootstrap starting.
I20260811 00:12:31.949565 11381 tablet_bootstrap.cc:654] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:31.950906 11381 log.cc:824] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:31.951057 11382 tablet_bootstrap.cc:492] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c: No bootstrap required, opened a new log
I20260811 00:12:31.951299 11382 ts_tablet_manager.cc:1397] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 00:12:31.951799 11383 tablet_bootstrap.cc:654] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:31.952962 11383 log.cc:824] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:31.954005 11381 tablet_bootstrap.cc:492] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b: No bootstrap required, opened a new log
I20260811 00:12:31.954313 11381 ts_tablet_manager.cc:1397] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 00:12:31.955942 11383 tablet_bootstrap.cc:492] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d: No bootstrap required, opened a new log
I20260811 00:12:31.956231 11383 ts_tablet_manager.cc:1397] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 00:12:31.959547 11382 raft_consensus.cc:348] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:31.960058 11382 raft_consensus.cc:374] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:31.960189 11382 raft_consensus.cc:729] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd9b2df5cede4dc19d32067269d7e50c, State: Initialized, Role: FOLLOWER
I20260811 00:12:31.960839 11382 consensus_queue.cc:260] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:31.961916 11121 heartbeater.cc:503] Master 127.7.94.62:33893 was elected leader, sending a full tablet report...
I20260811 00:12:31.962566 11382 ts_tablet_manager.cc:1428] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260811 00:12:31.964944 11381 raft_consensus.cc:348] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:31.965425 11381 raft_consensus.cc:374] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:31.965538 11381 raft_consensus.cc:729] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18642798f024415cba62deab3184f14b, State: Initialized, Role: FOLLOWER
I20260811 00:12:31.966185 11381 consensus_queue.cc:260] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:31.966729 11383 raft_consensus.cc:348] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:31.967233 11383 raft_consensus.cc:374] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:31.967373 11383 raft_consensus.cc:729] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89b899be3f3b4dec819fdc0e2230e35d, State: Initialized, Role: FOLLOWER
I20260811 00:12:31.968138 11381 ts_tablet_manager.cc:1428] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b: Time spent starting tablet: real 0.014s user 0.002s sys 0.013s
I20260811 00:12:31.968151 11383 consensus_queue.cc:260] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:31.969947 11347 heartbeater.cc:503] Master 127.7.94.62:33893 was elected leader, sending a full tablet report...
I20260811 00:12:31.970475 11383 ts_tablet_manager.cc:1428] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d: Time spent starting tablet: real 0.014s user 0.007s sys 0.006s
I20260811 00:12:31.976349 11392 raft_consensus.cc:482] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:12:31.976570 11392 raft_consensus.cc:504] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:31.978057 11392 leader_election.cc:288] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd9b2df5cede4dc19d32067269d7e50c (127.7.94.2:34595), 89b899be3f3b4dec819fdc0e2230e35d (127.7.94.3:44257)
I20260811 00:12:31.984784 11374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:31.978467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:36739 (local address 127.7.94.2:34595)
0811 00:12:31.978658 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:31.978662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:31.979240 (+ 578us) server_negotiation.cc:407] Connection header received
0811 00:12:31.979575 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:31.979581 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:31.979659 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:31.979716 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:31.980413 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.981157 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.983127 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.983872 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.984021 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:31.984141 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:31.984396 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 00:12:31.984518 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 00:12:31.984912 11380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:31.979426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:37229 (local address 127.7.94.3:44257)
0811 00:12:31.979637 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:31.979644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:31.979888 (+ 244us) server_negotiation.cc:407] Connection header received
0811 00:12:31.980097 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:31.980102 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:31.980161 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:31.980248 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:31.980762 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.981514 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.983297 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.984032 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.984179 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:31.984292 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:31.984543 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 00:12:31.984669 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:12:31.984961 11394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:31.978434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:34595 (local address 127.7.94.1:36739)
0811 00:12:31.979146 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:31.979166 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:31.979359 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:31.979913 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:31.979916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:31.979961 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:31.980279 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:31.980287 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.981359 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.981368 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:31.982981 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:31.982993 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.984076 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.984082 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:31.984245 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:31.984268 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 00:12:31.984370 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 00:12:31.984495 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"mutex_wait_us":86,"spinlock_wait_cycles":16512,"thread_start_us":380,"threads_started":1}
I20260811 00:12:31.985078 11395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:31.979227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:44257 (local address 127.7.94.1:37229)
0811 00:12:31.979777 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:31.979809 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:31.979962 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:31.980272 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:31.980277 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:31.980301 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:31.980623 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:31.980632 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.981648 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.981653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:31.983153 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:31.983165 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:31.984254 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:31.984260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:31.984397 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:31.984420 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 00:12:31.984518 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 00:12:31.984644 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":17,"spinlock_wait_cycles":1792,"thread_start_us":273,"threads_started":1}
I20260811 00:12:31.986042 11258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "159f7eed403c444194251aab51e55582" candidate_uuid: "18642798f024415cba62deab3184f14b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89b899be3f3b4dec819fdc0e2230e35d" is_pre_election: true
I20260811 00:12:31.986048 11019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "159f7eed403c444194251aab51e55582" candidate_uuid: "18642798f024415cba62deab3184f14b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd9b2df5cede4dc19d32067269d7e50c" is_pre_election: true
I20260811 00:12:31.986503 11019 raft_consensus.cc:2454] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18642798f024415cba62deab3184f14b in term 0.
I20260811 00:12:31.986505 11258 raft_consensus.cc:2454] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18642798f024415cba62deab3184f14b in term 0.
I20260811 00:12:31.987133 10673 leader_election.cc:302] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18642798f024415cba62deab3184f14b, 89b899be3f3b4dec819fdc0e2230e35d; no voters:
I20260811 00:12:31.987491 11392 raft_consensus.cc:2781] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:12:31.987625 11392 raft_consensus.cc:482] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:12:31.987741 11392 raft_consensus.cc:3037] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:31.990393 11392 raft_consensus.cc:504] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:31.990955 11392 leader_election.cc:288] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [CANDIDATE]: Term 1 election: Requested vote from peers bd9b2df5cede4dc19d32067269d7e50c (127.7.94.2:34595), 89b899be3f3b4dec819fdc0e2230e35d (127.7.94.3:44257)
I20260811 00:12:31.991487 11019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "159f7eed403c444194251aab51e55582" candidate_uuid: "18642798f024415cba62deab3184f14b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd9b2df5cede4dc19d32067269d7e50c"
I20260811 00:12:31.991528 11258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "159f7eed403c444194251aab51e55582" candidate_uuid: "18642798f024415cba62deab3184f14b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89b899be3f3b4dec819fdc0e2230e35d"
I20260811 00:12:31.991717 11019 raft_consensus.cc:3037] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:31.991760 11258 raft_consensus.cc:3037] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:31.994084 11019 raft_consensus.cc:2454] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18642798f024415cba62deab3184f14b in term 1.
I20260811 00:12:31.994225 11258 raft_consensus.cc:2454] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18642798f024415cba62deab3184f14b in term 1.
I20260811 00:12:31.994560 10673 leader_election.cc:302] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18642798f024415cba62deab3184f14b, bd9b2df5cede4dc19d32067269d7e50c; no voters:
I20260811 00:12:31.994874 11392 raft_consensus.cc:2781] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:31.995359 11392 raft_consensus.cc:686] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [term 1 LEADER]: Becoming Leader. State: Replica: 18642798f024415cba62deab3184f14b, State: Running, Role: LEADER
I20260811 00:12:31.995808 11392 consensus_queue.cc:237] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:32.000563 10527 catalog_manager.cc:5495] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b reported cstate change: term changed from 0 to 1, leader changed from <none> to 18642798f024415cba62deab3184f14b (127.7.94.1). New cstate: current_term: 1 leader_uuid: "18642798f024415cba62deab3184f14b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } health_report { overall_health: UNKNOWN } } }
W20260811 00:12:32.021806 10873 tablet.cc:2369] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 00:12:32 dist-test-slave-8hhm krb5kdc[10454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407150, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:32.052116 11411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.039883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:33893 (local address 127.0.0.1:48798)
0811 00:12:32.040497 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:32.040512 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:32.040637 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:32.040926 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:32.040929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:32.041637 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:32.041912 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:32.041921 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.042863 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.042866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:32.043796 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:32.043806 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.044452 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.044456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:32.044574 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.045168 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:32.045189 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:32.047318 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:32.050045 (+ 2727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:32.050055 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:32.050060 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:32.050439 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:32.050756 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:32.050759 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:32.050761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:32.050887 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:32.051328 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:32.051334 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:32.051472 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 00:12:32.051598 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 00:12:32.051805 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":368,"threads_started":1}
I20260811 00:12:32.052194 10914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.039822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48798 (local address 127.7.94.62:33893)
0811 00:12:32.040019 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:32.040025 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:32.040601 (+ 576us) server_negotiation.cc:407] Connection header received
0811 00:12:32.040742 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:32.040745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:32.040802 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:32.040880 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:32.042040 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.042751 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.043905 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.044287 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.044371 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.047488 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:32.047493 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:32.047496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:32.047522 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:32.049927 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:32.050534 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:32.050538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:32.050540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:32.050669 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:32.050972 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:32.050975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:32.050977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:32.051223 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:32.051349 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:32.051715 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 00:12:32.051902 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":58}
I20260811 00:12:32.056941 10527 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:48798:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 00:12:32.058012 10527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:12:32.064320 10991 tablet_service.cc:1467] Processing CreateTablet for tablet 85e624e8b52f48d4a678e7288955906d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e7ae8a6f7bcf4a93ac7e422955365b69]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:12:32.064320 10751 tablet_service.cc:1467] Processing CreateTablet for tablet 85e624e8b52f48d4a678e7288955906d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e7ae8a6f7bcf4a93ac7e422955365b69]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:12:32.064528 11228 tablet_service.cc:1467] Processing CreateTablet for tablet 85e624e8b52f48d4a678e7288955906d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e7ae8a6f7bcf4a93ac7e422955365b69]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:12:32.065362 11228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85e624e8b52f48d4a678e7288955906d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:32.065361 10751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85e624e8b52f48d4a678e7288955906d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:32.065361 10991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85e624e8b52f48d4a678e7288955906d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:32.071403 11383 tablet_bootstrap.cc:492] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d: Bootstrap starting.
W20260811 00:12:32.071797 11122 tablet.cc:2369] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:12:32.072547 11382 tablet_bootstrap.cc:492] T 85e624e8b52f48d4a678e7288955906d P bd9b2df5cede4dc19d32067269d7e50c: Bootstrap starting.
I20260811 00:12:32.072543 11381 tablet_bootstrap.cc:492] T 85e624e8b52f48d4a678e7288955906d P 18642798f024415cba62deab3184f14b: Bootstrap starting.
W20260811 00:12:32.072813 11349 tablet.cc:2369] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:12:32.073199 11383 tablet_bootstrap.cc:654] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:32.074239 11382 tablet_bootstrap.cc:654] T 85e624e8b52f48d4a678e7288955906d P bd9b2df5cede4dc19d32067269d7e50c: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:32.074239 11381 tablet_bootstrap.cc:654] T 85e624e8b52f48d4a678e7288955906d P 18642798f024415cba62deab3184f14b: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:32.075619 11383 tablet_bootstrap.cc:492] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d: No bootstrap required, opened a new log
I20260811 00:12:32.075793 11383 ts_tablet_manager.cc:1397] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 00:12:32.076411 11383 raft_consensus.cc:348] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:32.076632 11383 raft_consensus.cc:374] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:32.076721 11383 raft_consensus.cc:729] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89b899be3f3b4dec819fdc0e2230e35d, State: Initialized, Role: FOLLOWER
I20260811 00:12:32.076967 11383 consensus_queue.cc:260] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:32.077587 11383 ts_tablet_manager.cc:1428] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 00:12:32.077517 11393 tablet_replica.cc:439] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } } }
I20260811 00:12:32.078006 11393 tablet_replica.cc:442] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:12:32.079166 11381 tablet_bootstrap.cc:492] T 85e624e8b52f48d4a678e7288955906d P 18642798f024415cba62deab3184f14b: No bootstrap required, opened a new log
I20260811 00:12:32.079315 11381 ts_tablet_manager.cc:1397] T 85e624e8b52f48d4a678e7288955906d P 18642798f024415cba62deab3184f14b: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 00:12:32.080137 11381 raft_consensus.cc:348] T 85e624e8b52f48d4a678e7288955906d P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:32.080340 11381 raft_consensus.cc:374] T 85e624e8b52f48d4a678e7288955906d P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:32.080440 11381 raft_consensus.cc:729] T 85e624e8b52f48d4a678e7288955906d P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18642798f024415cba62deab3184f14b, State: Initialized, Role: FOLLOWER
I20260811 00:12:32.080682 11381 consensus_queue.cc:260] T 85e624e8b52f48d4a678e7288955906d P 18642798f024415cba62deab3184f14b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:32.081281 11381 ts_tablet_manager.cc:1428] T 85e624e8b52f48d4a678e7288955906d P 18642798f024415cba62deab3184f14b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 00:12:32.081084 11392 tablet_replica.cc:439] T 85e624e8b52f48d4a678e7288955906d P 18642798f024415cba62deab3184f14b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } } }
I20260811 00:12:32.081938 11392 tablet_replica.cc:442] T 85e624e8b52f48d4a678e7288955906d P 18642798f024415cba62deab3184f14b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:12:32.086102 11382 tablet_bootstrap.cc:492] T 85e624e8b52f48d4a678e7288955906d P bd9b2df5cede4dc19d32067269d7e50c: No bootstrap required, opened a new log
I20260811 00:12:32.086272 11382 ts_tablet_manager.cc:1397] T 85e624e8b52f48d4a678e7288955906d P bd9b2df5cede4dc19d32067269d7e50c: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.010s
I20260811 00:12:32.086897 11382 raft_consensus.cc:348] T 85e624e8b52f48d4a678e7288955906d P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:32.087077 11382 raft_consensus.cc:374] T 85e624e8b52f48d4a678e7288955906d P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:32.087198 11382 raft_consensus.cc:729] T 85e624e8b52f48d4a678e7288955906d P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd9b2df5cede4dc19d32067269d7e50c, State: Initialized, Role: FOLLOWER
I20260811 00:12:32.087427 11382 consensus_queue.cc:260] T 85e624e8b52f48d4a678e7288955906d P bd9b2df5cede4dc19d32067269d7e50c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:32.087988 11382 ts_tablet_manager.cc:1428] T 85e624e8b52f48d4a678e7288955906d P bd9b2df5cede4dc19d32067269d7e50c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 00:12:32.087807 11389 tablet_replica.cc:439] T 85e624e8b52f48d4a678e7288955906d P bd9b2df5cede4dc19d32067269d7e50c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } } }
I20260811 00:12:32.088164 11389 tablet_replica.cc:442] T 85e624e8b52f48d4a678e7288955906d P bd9b2df5cede4dc19d32067269d7e50c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:12:32.239677 11393 raft_consensus.cc:482] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:12:32.239908 11393 raft_consensus.cc:504] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:32.241559 11393 leader_election.cc:288] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18642798f024415cba62deab3184f14b (127.7.94.1:37555), bd9b2df5cede4dc19d32067269d7e50c (127.7.94.2:34595)
I20260811 00:12:32.247812 11162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.241735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:37555 (local address 127.7.94.3:50773)
0811 00:12:32.241987 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:32.242012 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:32.242151 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:32.242670 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:32.242677 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:32.242715 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:32.242945 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:32.242958 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.244156 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.244160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:32.246039 (+ 1879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:32.246055 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.247277 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.247280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:32.247412 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.247435 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 00:12:32.247506 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 00:12:32.247593 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":23}
I20260811 00:12:32.247951 11378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.241862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:50773 (local address 127.7.94.1:37555)
0811 00:12:32.242051 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:32.242059 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:32.242081 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:12:32.242367 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:32.242376 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:32.242491 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:32.242619 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:32.243082 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.244020 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.246201 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.247131 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.247262 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.247360 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:32.247565 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 00:12:32.247707 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 00:12:32.248860 11416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.242120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:34595 (local address 127.7.94.3:39097)
0811 00:12:32.242846 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:32.242870 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:32.242994 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:32.243328 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:32.243331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:32.243350 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:32.243657 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:32.243665 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.244801 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.244807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:32.246613 (+ 1806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:32.246623 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.248225 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.248231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:32.248394 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.248429 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 00:12:32.248508 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 00:12:32.248587 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"mutex_wait_us":4,"thread_start_us":403,"threads_started":1}
I20260811 00:12:32.248941 11374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.242116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:39097 (local address 127.7.94.2:34595)
0811 00:12:32.242322 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:32.242333 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:32.242965 (+ 632us) server_negotiation.cc:407] Connection header received
0811 00:12:32.243135 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:32.243141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:32.243214 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:32.243309 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:32.243805 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.244657 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.246754 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.247915 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.248040 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.248141 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:32.248561 (+ 420us) server_negotiation.cc:299] Negotiation successful
0811 00:12:32.248689 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:12:32.248860 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85e624e8b52f48d4a678e7288955906d" candidate_uuid: "89b899be3f3b4dec819fdc0e2230e35d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18642798f024415cba62deab3184f14b" is_pre_election: true
I20260811 00:12:32.249388 10794 raft_consensus.cc:2454] T 85e624e8b52f48d4a678e7288955906d P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89b899be3f3b4dec819fdc0e2230e35d in term 0.
I20260811 00:12:32.249619 11019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85e624e8b52f48d4a678e7288955906d" candidate_uuid: "89b899be3f3b4dec819fdc0e2230e35d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd9b2df5cede4dc19d32067269d7e50c" is_pre_election: true
I20260811 00:12:32.249871 11019 raft_consensus.cc:2454] T 85e624e8b52f48d4a678e7288955906d P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89b899be3f3b4dec819fdc0e2230e35d in term 0.
I20260811 00:12:32.250023 11146 leader_election.cc:302] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18642798f024415cba62deab3184f14b, 89b899be3f3b4dec819fdc0e2230e35d; no voters:
I20260811 00:12:32.250444 11393 raft_consensus.cc:2781] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:12:32.250603 11393 raft_consensus.cc:482] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:12:32.250718 11393 raft_consensus.cc:3037] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:32.252810 11393 raft_consensus.cc:504] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:32.253440 11393 leader_election.cc:288] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d [CANDIDATE]: Term 1 election: Requested vote from peers 18642798f024415cba62deab3184f14b (127.7.94.1:37555), bd9b2df5cede4dc19d32067269d7e50c (127.7.94.2:34595)
I20260811 00:12:32.253821 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85e624e8b52f48d4a678e7288955906d" candidate_uuid: "89b899be3f3b4dec819fdc0e2230e35d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18642798f024415cba62deab3184f14b"
I20260811 00:12:32.253901 11019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85e624e8b52f48d4a678e7288955906d" candidate_uuid: "89b899be3f3b4dec819fdc0e2230e35d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd9b2df5cede4dc19d32067269d7e50c"
I20260811 00:12:32.254006 10794 raft_consensus.cc:3037] T 85e624e8b52f48d4a678e7288955906d P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:32.254124 11019 raft_consensus.cc:3037] T 85e624e8b52f48d4a678e7288955906d P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:32.256594 11019 raft_consensus.cc:2454] T 85e624e8b52f48d4a678e7288955906d P bd9b2df5cede4dc19d32067269d7e50c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89b899be3f3b4dec819fdc0e2230e35d in term 1.
I20260811 00:12:32.256601 10794 raft_consensus.cc:2454] T 85e624e8b52f48d4a678e7288955906d P 18642798f024415cba62deab3184f14b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89b899be3f3b4dec819fdc0e2230e35d in term 1.
I20260811 00:12:32.256978 11146 leader_election.cc:302] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18642798f024415cba62deab3184f14b, 89b899be3f3b4dec819fdc0e2230e35d; no voters:
I20260811 00:12:32.257279 11393 raft_consensus.cc:2781] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:32.257752 11393 raft_consensus.cc:686] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d [term 1 LEADER]: Becoming Leader. State: Replica: 89b899be3f3b4dec819fdc0e2230e35d, State: Running, Role: LEADER
I20260811 00:12:32.258241 11393 consensus_queue.cc:237] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:32.261534 11417 tablet_replica.cc:439] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d: TxnStatusTablet state changed. Reason: New leader 89b899be3f3b4dec819fdc0e2230e35d. Latest consensus state: current_term: 1 leader_uuid: "89b899be3f3b4dec819fdc0e2230e35d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } } }
I20260811 00:12:32.261874 11417 tablet_replica.cc:442] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:12:32.262573 11419 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 00:12:32.262825 11419 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 00:12:32.263386 10527 catalog_manager.cc:5495] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d reported cstate change: term changed from 0 to 1, leader changed from <none> to 89b899be3f3b4dec819fdc0e2230e35d (127.7.94.3). New cstate: current_term: 1 leader_uuid: "89b899be3f3b4dec819fdc0e2230e35d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } health_report { overall_health: HEALTHY } } }
I20260811 00:12:32.317444 11411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.310755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:44257 (local address 127.0.0.1:44096)
0811 00:12:32.310990 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:32.311011 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:32.311135 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:32.311668 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:32.311671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:32.311681 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:12:32.311841 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:32.311848 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.313320 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.313323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:32.314449 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:32.314462 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.315493 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.315501 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:32.315771 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.315852 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:12:32.316854 (+ 1002us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:12:32.316875 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 00:12:32.316966 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 00:12:32.317128 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 00:12:32.317443 11380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.310768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44096 (local address 127.7.94.3:44257)
0811 00:12:32.310969 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:32.310976 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:32.311087 (+ 111us) server_negotiation.cc:407] Connection header received
0811 00:12:32.311337 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:32.311343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:32.311481 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:32.311644 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:12:32.311961 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.313173 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.314617 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.315336 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.315481 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.316069 (+ 588us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:12:32.316732 (+ 663us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:12:32.316869 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:32.317006 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 00:12:32.317101 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 00:12:32.326583 11019 raft_consensus.cc:1261] T 85e624e8b52f48d4a678e7288955906d P bd9b2df5cede4dc19d32067269d7e50c [term 1 FOLLOWER]: Refusing update from remote peer 89b899be3f3b4dec819fdc0e2230e35d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:12:32.327849 10794 raft_consensus.cc:1261] T 85e624e8b52f48d4a678e7288955906d P 18642798f024415cba62deab3184f14b [term 1 FOLLOWER]: Refusing update from remote peer 89b899be3f3b4dec819fdc0e2230e35d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:12:32.328632 11393 consensus_queue.cc:1035] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:12:32.329329 11393 consensus_queue.cc:1035] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d [LEADER]: Connected to new peer: Peer: permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:12:32.337688 11389 tablet_replica.cc:439] T 85e624e8b52f48d4a678e7288955906d P bd9b2df5cede4dc19d32067269d7e50c: TxnStatusTablet state changed. Reason: New leader 89b899be3f3b4dec819fdc0e2230e35d. Latest consensus state: current_term: 1 leader_uuid: "89b899be3f3b4dec819fdc0e2230e35d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } } }
I20260811 00:12:32.337971 11389 tablet_replica.cc:442] T 85e624e8b52f48d4a678e7288955906d P bd9b2df5cede4dc19d32067269d7e50c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:12:32.340003 11424 mvcc.cc:205] Tried to move back new op lower bound from 7317123695919509504 to 7317123695653785600. Current Snapshot: MvccSnapshot[applied={T|T < 7317123695919509504}]
I20260811 00:12:32.345633 11417 tablet_replica.cc:439] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "89b899be3f3b4dec819fdc0e2230e35d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } } }
I20260811 00:12:32.345890 11417 tablet_replica.cc:442] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:12:32.354358 11389 tablet_replica.cc:439] T 85e624e8b52f48d4a678e7288955906d P bd9b2df5cede4dc19d32067269d7e50c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "89b899be3f3b4dec819fdc0e2230e35d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } } }
I20260811 00:12:32.354629 11389 tablet_replica.cc:442] T 85e624e8b52f48d4a678e7288955906d P bd9b2df5cede4dc19d32067269d7e50c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:12:32.362871 11392 tablet_replica.cc:439] T 85e624e8b52f48d4a678e7288955906d P 18642798f024415cba62deab3184f14b: TxnStatusTablet state changed. Reason: New leader 89b899be3f3b4dec819fdc0e2230e35d. Latest consensus state: current_term: 1 leader_uuid: "89b899be3f3b4dec819fdc0e2230e35d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } } }
I20260811 00:12:32.363271 11392 tablet_replica.cc:442] T 85e624e8b52f48d4a678e7288955906d P 18642798f024415cba62deab3184f14b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:12:32.364856 11393 tablet_replica.cc:439] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "89b899be3f3b4dec819fdc0e2230e35d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } } }
I20260811 00:12:32.365168 11393 tablet_replica.cc:442] T 85e624e8b52f48d4a678e7288955906d P 89b899be3f3b4dec819fdc0e2230e35d: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:12:32.368547 11392 tablet_replica.cc:439] T 85e624e8b52f48d4a678e7288955906d P 18642798f024415cba62deab3184f14b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "89b899be3f3b4dec819fdc0e2230e35d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } } }
I20260811 00:12:32.368873 11392 tablet_replica.cc:442] T 85e624e8b52f48d4a678e7288955906d P 18642798f024415cba62deab3184f14b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:12:32.371950 11378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.362361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36358 (local address 127.7.94.1:37555)
0811 00:12:32.362548 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:32.362555 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:32.363230 (+ 675us) server_negotiation.cc:407] Connection header received
0811 00:12:32.363352 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:32.363358 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:32.363491 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:32.363657 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:12:32.364248 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.365351 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.367858 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.368607 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.369297 (+ 690us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.369440 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:12:32.371309 (+ 1869us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:12:32.371465 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:32.371548 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 00:12:32.371632 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 00:12:32.387951 11380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.381844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:48793 (local address 127.7.94.3:44257)
0811 00:12:32.382049 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:32.382058 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:32.382682 (+ 624us) server_negotiation.cc:407] Connection header received
0811 00:12:32.382870 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:32.382875 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:32.382972 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:32.383051 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:12:32.383606 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.384305 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.386340 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.386756 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.386847 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.387366 (+ 519us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:12:32.387456 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:12:32.387519 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:32.387675 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 00:12:32.387739 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 00:12:32.388176 11451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.381865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:44257 (local address 127.7.94.1:48793)
0811 00:12:32.382579 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:32.382599 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:32.382756 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:32.383093 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:32.383099 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:32.383115 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:12:32.383464 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:32.383475 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.384448 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.384454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:32.386210 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:32.386226 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.386928 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.386934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:32.387110 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.387183 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:12:32.387547 (+ 364us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:12:32.387560 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 00:12:32.387638 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 00:12:32.387790 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":364,"threads_started":1}
I20260811 00:12:32.398518 11019 raft_consensus.cc:1261] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c [term 1 FOLLOWER]: Refusing update from remote peer 18642798f024415cba62deab3184f14b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:12:32.398586 11258 raft_consensus.cc:1261] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d [term 1 FOLLOWER]: Refusing update from remote peer 18642798f024415cba62deab3184f14b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:12:32.399189 11398 consensus_queue.cc:1035] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:12:32.399838 11392 consensus_queue.cc:1035] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [LEADER]: Connected to new peer: Peer: permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:12:32.410256 11428 mvcc.cc:205] Tried to move back new op lower bound from 7317123696217743360 to 7317123694578864128. Current Snapshot: MvccSnapshot[applied={T|T < 7317123696217743360}]
I20260811 00:12:32.451083 11378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.434792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:45009 (local address 127.7.94.1:37555)
0811 00:12:32.434976 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:32.434982 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:32.445465 (+ 10483us) server_negotiation.cc:407] Connection header received
0811 00:12:32.445633 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:32.445638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:32.445731 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:32.445838 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:12:32.446312 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.447458 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.449294 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.449749 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.449882 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.450340 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:12:32.450451 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:12:32.450544 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:32.450687 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 00:12:32.450769 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 00:12:32.451206 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.434722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:37555 (local address 127.7.94.3:45009)
0811 00:12:32.445359 (+ 10637us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:32.445379 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:32.445504 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:32.445842 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:32.445845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:32.445854 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:12:32.446167 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:32.446174 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.447625 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.447634 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:32.449062 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:32.449080 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.449947 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.449954 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:32.450092 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.450153 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:12:32.450553 (+ 400us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:12:32.450566 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 00:12:32.450663 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 00:12:32.450798 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10412,"thread_start_us":10853,"threads_started":1}
I20260811 00:12:32.490245 10529 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:12:32.490531 10529 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:12:32.494549 10529 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc: Sending DeleteTablet for 3 replicas of tablet 159f7eed403c444194251aab51e55582
I20260811 00:12:32.495760 10991 tablet_service.cc:1514] Processing DeleteTablet for tablet 159f7eed403c444194251aab51e55582 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:32 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:48608
I20260811 00:12:32.496658 11465 tablet_replica.cc:330] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c: stopping tablet replica
I20260811 00:12:32.497169 11465 raft_consensus.cc:2229] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:12:32.497321 10751 tablet_service.cc:1514] Processing DeleteTablet for tablet 159f7eed403c444194251aab51e55582 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:32 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:36346
I20260811 00:12:32.497648 11465 raft_consensus.cc:2258] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:32.499306 11228 tablet_service.cc:1514] Processing DeleteTablet for tablet 159f7eed403c444194251aab51e55582 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:32 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:44082
I20260811 00:12:32.499639 10529 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:12:32.500308 10529 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:12:32.507439 10750 tablet_service.cc:1467] Processing CreateTablet for tablet 4db557a52598476b957f53076628eb70 (DEFAULT_TABLE table=test-table [id=a702fb18fc6b481aa437e8df36cc9329]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:32.507963 10750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db557a52598476b957f53076628eb70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:32.507839 11227 tablet_service.cc:1467] Processing CreateTablet for tablet 4db557a52598476b957f53076628eb70 (DEFAULT_TABLE table=test-table [id=a702fb18fc6b481aa437e8df36cc9329]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:32.508433 11227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db557a52598476b957f53076628eb70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:32.509936 10991 tablet_service.cc:1467] Processing CreateTablet for tablet 4db557a52598476b957f53076628eb70 (DEFAULT_TABLE table=test-table [id=a702fb18fc6b481aa437e8df36cc9329]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:32.515591 11381 tablet_bootstrap.cc:492] T 4db557a52598476b957f53076628eb70 P 18642798f024415cba62deab3184f14b: Bootstrap starting.
I20260811 00:12:32.517412 11381 tablet_bootstrap.cc:654] T 4db557a52598476b957f53076628eb70 P 18642798f024415cba62deab3184f14b: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:32.519531 11383 tablet_bootstrap.cc:492] T 4db557a52598476b957f53076628eb70 P 89b899be3f3b4dec819fdc0e2230e35d: Bootstrap starting.
I20260811 00:12:32.521126 11383 tablet_bootstrap.cc:654] T 4db557a52598476b957f53076628eb70 P 89b899be3f3b4dec819fdc0e2230e35d: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:32.524077 10991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db557a52598476b957f53076628eb70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:32.533022 11382 tablet_bootstrap.cc:492] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c: Bootstrap starting.
I20260811 00:12:32.533334 11465 ts_tablet_manager.cc:1905] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:32.534993 11382 tablet_bootstrap.cc:654] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:32.548127 11465 ts_tablet_manager.cc:1918] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:12:32.548310 11465 log.cc:1196] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/159f7eed403c444194251aab51e55582
I20260811 00:12:32.548867 11465 ts_tablet_manager.cc:1939] T 159f7eed403c444194251aab51e55582 P bd9b2df5cede4dc19d32067269d7e50c: Deleting consensus metadata
I20260811 00:12:32.555817 11382 tablet_bootstrap.cc:492] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c: No bootstrap required, opened a new log
I20260811 00:12:32.556224 11382 ts_tablet_manager.cc:1397] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.010s
I20260811 00:12:32.557331 11382 raft_consensus.cc:348] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:32.557750 11382 raft_consensus.cc:374] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:32.557898 11382 raft_consensus.cc:729] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd9b2df5cede4dc19d32067269d7e50c, State: Initialized, Role: FOLLOWER
I20260811 00:12:32.557883 10501 catalog_manager.cc:4841] TS bd9b2df5cede4dc19d32067269d7e50c (127.7.94.2:34595): tablet 159f7eed403c444194251aab51e55582 (table test-table [id=0068d79ecdd84ec5bb3695dea857c577]) successfully deleted
I20260811 00:12:32.558566 11382 consensus_queue.cc:260] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:32.562067 11382 ts_tablet_manager.cc:1428] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260811 00:12:32.563580 11383 tablet_bootstrap.cc:492] T 4db557a52598476b957f53076628eb70 P 89b899be3f3b4dec819fdc0e2230e35d: No bootstrap required, opened a new log
I20260811 00:12:32.563848 11383 ts_tablet_manager.cc:1397] T 4db557a52598476b957f53076628eb70 P 89b899be3f3b4dec819fdc0e2230e35d: Time spent bootstrapping tablet: real 0.044s user 0.009s sys 0.003s
I20260811 00:12:32.564857 11383 raft_consensus.cc:348] T 4db557a52598476b957f53076628eb70 P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:32.565261 11383 raft_consensus.cc:374] T 4db557a52598476b957f53076628eb70 P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:32.565421 11383 raft_consensus.cc:729] T 4db557a52598476b957f53076628eb70 P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89b899be3f3b4dec819fdc0e2230e35d, State: Initialized, Role: FOLLOWER
I20260811 00:12:32.565826 11383 consensus_queue.cc:260] T 4db557a52598476b957f53076628eb70 P 89b899be3f3b4dec819fdc0e2230e35d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:32.566613 11383 ts_tablet_manager.cc:1428] T 4db557a52598476b957f53076628eb70 P 89b899be3f3b4dec819fdc0e2230e35d: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 00:12:32.569341 11466 tablet_replica.cc:330] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b: stopping tablet replica
I20260811 00:12:32.569909 11466 raft_consensus.cc:2229] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:12:32.570763 11466 raft_consensus.cc:2258] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:32.572057 11468 tablet_replica.cc:330] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d: stopping tablet replica
I20260811 00:12:32.575790 11381 tablet_bootstrap.cc:492] T 4db557a52598476b957f53076628eb70 P 18642798f024415cba62deab3184f14b: No bootstrap required, opened a new log
I20260811 00:12:32.576328 11381 ts_tablet_manager.cc:1397] T 4db557a52598476b957f53076628eb70 P 18642798f024415cba62deab3184f14b: Time spent bootstrapping tablet: real 0.060s user 0.013s sys 0.015s
I20260811 00:12:32.576344 11468 raft_consensus.cc:2229] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:12:32.576745 11468 raft_consensus.cc:2258] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:32.577200 11381 raft_consensus.cc:348] T 4db557a52598476b957f53076628eb70 P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:32.577737 11381 raft_consensus.cc:374] T 4db557a52598476b957f53076628eb70 P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:32.578179 11381 raft_consensus.cc:729] T 4db557a52598476b957f53076628eb70 P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18642798f024415cba62deab3184f14b, State: Initialized, Role: FOLLOWER
I20260811 00:12:32.578563 11466 ts_tablet_manager.cc:1905] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:32.578771 11468 ts_tablet_manager.cc:1905] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:32.578469 11381 consensus_queue.cc:260] T 4db557a52598476b957f53076628eb70 P 18642798f024415cba62deab3184f14b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:32.579380 11381 ts_tablet_manager.cc:1428] T 4db557a52598476b957f53076628eb70 P 18642798f024415cba62deab3184f14b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 00:12:32.586865 11468 ts_tablet_manager.cc:1918] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:12:32.587067 11468 log.cc:1196] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/159f7eed403c444194251aab51e55582
I20260811 00:12:32.587673 11468 ts_tablet_manager.cc:1939] T 159f7eed403c444194251aab51e55582 P 89b899be3f3b4dec819fdc0e2230e35d: Deleting consensus metadata
I20260811 00:12:32.587661 11466 ts_tablet_manager.cc:1918] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:12:32.587984 11466 log.cc:1196] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/159f7eed403c444194251aab51e55582
I20260811 00:12:32.588706 11466 ts_tablet_manager.cc:1939] T 159f7eed403c444194251aab51e55582 P 18642798f024415cba62deab3184f14b: Deleting consensus metadata
I20260811 00:12:32.589496 10501 catalog_manager.cc:4841] TS 89b899be3f3b4dec819fdc0e2230e35d (127.7.94.3:44257): tablet 159f7eed403c444194251aab51e55582 (table test-table [id=0068d79ecdd84ec5bb3695dea857c577]) successfully deleted
I20260811 00:12:32.590847 10498 catalog_manager.cc:4841] TS 18642798f024415cba62deab3184f14b (127.7.94.1:37555): tablet 159f7eed403c444194251aab51e55582 (table test-table [id=0068d79ecdd84ec5bb3695dea857c577]) successfully deleted
I20260811 00:12:32.593912 11389 raft_consensus.cc:482] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:12:32.594112 11389 raft_consensus.cc:504] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:32.595890 11389 leader_election.cc:288] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18642798f024415cba62deab3184f14b (127.7.94.1:37555), 89b899be3f3b4dec819fdc0e2230e35d (127.7.94.3:44257)
I20260811 00:12:32.615768 11378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.596718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:57349 (local address 127.7.94.1:37555)
0811 00:12:32.596907 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:32.596913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:32.606687 (+ 9774us) server_negotiation.cc:407] Connection header received
0811 00:12:32.607274 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:32.607281 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:32.607365 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:32.607440 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:32.608403 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.609685 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.612958 (+ 3273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.614415 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.614527 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.614628 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:32.615276 (+ 648us) server_negotiation.cc:299] Negotiation successful
0811 00:12:32.615428 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 00:12:32.616597 11476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.596321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:37555 (local address 127.7.94.2:57349)
0811 00:12:32.606560 (+ 10239us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:32.606585 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:32.607046 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:32.607705 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:32.607710 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:32.607741 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:32.608182 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:32.608199 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.609908 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.609914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:32.612274 (+ 2360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:32.612295 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.614877 (+ 2582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.614884 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:32.615085 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.615119 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 00:12:32.616063 (+ 944us) client_negotiation.cc:240] Negotiation successful
0811 00:12:32.616210 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10064,"thread_start_us":16311,"threads_started":1}
I20260811 00:12:32.617475 10789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db557a52598476b957f53076628eb70" candidate_uuid: "bd9b2df5cede4dc19d32067269d7e50c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18642798f024415cba62deab3184f14b" is_pre_election: true
I20260811 00:12:32.617769 10789 raft_consensus.cc:2454] T 4db557a52598476b957f53076628eb70 P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd9b2df5cede4dc19d32067269d7e50c in term 0.
I20260811 00:12:32.618362 10902 leader_election.cc:302] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18642798f024415cba62deab3184f14b, bd9b2df5cede4dc19d32067269d7e50c; no voters:
I20260811 00:12:32.618790 11389 raft_consensus.cc:2781] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:12:32.618916 11389 raft_consensus.cc:482] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:12:32.618997 11389 raft_consensus.cc:3037] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:32.622273 11389 raft_consensus.cc:504] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:32.622951 11389 leader_election.cc:288] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [CANDIDATE]: Term 1 election: Requested vote from peers 18642798f024415cba62deab3184f14b (127.7.94.1:37555), 89b899be3f3b4dec819fdc0e2230e35d (127.7.94.3:44257)
I20260811 00:12:32.623577 10789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db557a52598476b957f53076628eb70" candidate_uuid: "bd9b2df5cede4dc19d32067269d7e50c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18642798f024415cba62deab3184f14b"
I20260811 00:12:32.623828 10789 raft_consensus.cc:3037] T 4db557a52598476b957f53076628eb70 P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:32.624677 11380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.596976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:36039 (local address 127.7.94.3:44257)
0811 00:12:32.597398 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:32.597410 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:32.613097 (+ 15687us) server_negotiation.cc:407] Connection header received
0811 00:12:32.613599 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:32.613606 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:32.613697 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:32.613775 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:32.614923 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.616130 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.621389 (+ 5259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.622897 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.623024 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.623150 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:32.624159 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0811 00:12:32.624298 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 00:12:32.625658 11474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.596499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:44257 (local address 127.7.94.2:36039)
0811 00:12:32.612976 (+ 16477us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:32.613002 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:32.613360 (+ 358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:32.614019 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:32.614024 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:32.614054 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:32.614619 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:32.614641 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.616862 (+ 2221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.616868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:32.621121 (+ 4253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:32.621140 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.623595 (+ 2455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.623602 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:32.623894 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.623978 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 00:12:32.625094 (+ 1116us) client_negotiation.cc:240] Negotiation successful
0811 00:12:32.625267 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16327,"thread_start_us":10200,"threads_started":1}
I20260811 00:12:32.626926 11257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db557a52598476b957f53076628eb70" candidate_uuid: "bd9b2df5cede4dc19d32067269d7e50c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89b899be3f3b4dec819fdc0e2230e35d" is_pre_election: true
I20260811 00:12:32.627213 10789 raft_consensus.cc:2454] T 4db557a52598476b957f53076628eb70 P 18642798f024415cba62deab3184f14b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd9b2df5cede4dc19d32067269d7e50c in term 1.
I20260811 00:12:32.627293 11257 raft_consensus.cc:2454] T 4db557a52598476b957f53076628eb70 P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd9b2df5cede4dc19d32067269d7e50c in term 0.
I20260811 00:12:32.627713 10902 leader_election.cc:302] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18642798f024415cba62deab3184f14b, bd9b2df5cede4dc19d32067269d7e50c; no voters:
I20260811 00:12:32.627759 11258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db557a52598476b957f53076628eb70" candidate_uuid: "bd9b2df5cede4dc19d32067269d7e50c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89b899be3f3b4dec819fdc0e2230e35d"
I20260811 00:12:32.627980 11258 raft_consensus.cc:3037] T 4db557a52598476b957f53076628eb70 P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:32.628011 11389 raft_consensus.cc:2781] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:32.632191 11258 raft_consensus.cc:2454] T 4db557a52598476b957f53076628eb70 P 89b899be3f3b4dec819fdc0e2230e35d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd9b2df5cede4dc19d32067269d7e50c in term 1.
I20260811 00:12:32.633488 11389 raft_consensus.cc:686] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [term 1 LEADER]: Becoming Leader. State: Replica: bd9b2df5cede4dc19d32067269d7e50c, State: Running, Role: LEADER
I20260811 00:12:32.633966 11389 consensus_queue.cc:237] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:32.638087 10527 catalog_manager.cc:5495] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c reported cstate change: term changed from 0 to 1, leader changed from <none> to bd9b2df5cede4dc19d32067269d7e50c (127.7.94.2). New cstate: current_term: 1 leader_uuid: "bd9b2df5cede4dc19d32067269d7e50c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } health_report { overall_health: UNKNOWN } } }
I20260811 00:12:32.668133 11374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.660486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48618 (local address 127.7.94.2:34595)
0811 00:12:32.660653 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:32.660660 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:32.660982 (+ 322us) server_negotiation.cc:407] Connection header received
0811 00:12:32.661281 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:32.661287 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:32.661418 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:32.661606 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:12:32.662078 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.663239 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.665255 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.665922 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.666056 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.666656 (+ 600us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:12:32.667426 (+ 770us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:12:32.667608 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:32.667743 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 00:12:32.667833 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 00:12:32.687994 11380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.680887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:55755 (local address 127.7.94.3:44257)
0811 00:12:32.681108 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:32.681115 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:32.681378 (+ 263us) server_negotiation.cc:407] Connection header received
0811 00:12:32.681553 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:32.681559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:32.681661 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:32.681770 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:12:32.682364 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.683531 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.685604 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.686378 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.686535 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.687131 (+ 596us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:12:32.687287 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:12:32.687401 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:32.687563 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 00:12:32.687652 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 00:12:32.688053 11485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.680471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:44257 (local address 127.7.94.2:55755)
0811 00:12:32.681194 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:32.681237 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:32.681398 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:32.681814 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:32.681819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:32.681833 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:12:32.682200 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:32.682213 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.683690 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.683696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:32.685425 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:32.685442 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.686608 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.686615 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:32.686815 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.686904 (+ 89us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:12:32.687412 (+ 508us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:12:32.687424 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 00:12:32.687519 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 00:12:32.687670 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":646,"threads_started":1}
I20260811 00:12:32.699976 11258 raft_consensus.cc:1261] T 4db557a52598476b957f53076628eb70 P 89b899be3f3b4dec819fdc0e2230e35d [term 1 FOLLOWER]: Refusing update from remote peer bd9b2df5cede4dc19d32067269d7e50c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:12:32.700049 10794 raft_consensus.cc:1261] T 4db557a52598476b957f53076628eb70 P 18642798f024415cba62deab3184f14b [term 1 FOLLOWER]: Refusing update from remote peer bd9b2df5cede4dc19d32067269d7e50c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:12:32.700862 11477 consensus_queue.cc:1035] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [LEADER]: Connected to new peer: Peer: permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:12:32.701606 11477 consensus_queue.cc:1035] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [LEADER]: Connected to new peer: Peer: permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:12:32.757778 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.750878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:34595 (local address 127.7.94.3:41015)
0811 00:12:32.751089 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:32.751112 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:32.751256 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:32.751956 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:32.751962 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:32.751976 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:12:32.752182 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:32.752193 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.753678 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.753684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:32.755359 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:32.755376 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.756517 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.756522 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:32.756712 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.756764 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:12:32.757204 (+ 440us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:12:32.757235 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 00:12:32.757310 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 00:12:32.757457 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 00:12:32.758421 11374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:32.751294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:41015 (local address 127.7.94.2:34595)
0811 00:12:32.751457 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:32.751463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:32.751481 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:12:32.751570 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:32.751575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:32.751660 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:32.751788 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:12:32.752368 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.753464 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.755545 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:32.756282 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:32.756407 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:32.756954 (+ 547us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:12:32.757085 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:12:32.757158 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:32.758102 (+ 944us) server_negotiation.cc:299] Negotiation successful
0811 00:12:32.758177 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 00:12:32.759793 11428 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 00:12:34.389724 10529 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:12:34.389986 10529 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:12:34.393889 10529 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc: Sending DeleteTablet for 3 replicas of tablet 4db557a52598476b957f53076628eb70
I20260811 00:12:34.394665 10991 tablet_service.cc:1514] Processing DeleteTablet for tablet 4db557a52598476b957f53076628eb70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:34 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:48608
I20260811 00:12:34.394714 10751 tablet_service.cc:1514] Processing DeleteTablet for tablet 4db557a52598476b957f53076628eb70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:34 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:36346
I20260811 00:12:34.394770 11228 tablet_service.cc:1514] Processing DeleteTablet for tablet 4db557a52598476b957f53076628eb70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:34 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:44082
I20260811 00:12:34.405682 11533 tablet_replica.cc:330] T 4db557a52598476b957f53076628eb70 P 89b899be3f3b4dec819fdc0e2230e35d: stopping tablet replica
I20260811 00:12:34.406050 11533 raft_consensus.cc:2229] T 4db557a52598476b957f53076628eb70 P 89b899be3f3b4dec819fdc0e2230e35d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:12:34.406399 11533 raft_consensus.cc:2258] T 4db557a52598476b957f53076628eb70 P 89b899be3f3b4dec819fdc0e2230e35d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:34.407011 11534 tablet_replica.cc:330] T 4db557a52598476b957f53076628eb70 P 18642798f024415cba62deab3184f14b: stopping tablet replica
I20260811 00:12:34.407328 11534 raft_consensus.cc:2229] T 4db557a52598476b957f53076628eb70 P 18642798f024415cba62deab3184f14b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:12:34.407615 11534 raft_consensus.cc:2258] T 4db557a52598476b957f53076628eb70 P 18642798f024415cba62deab3184f14b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:34.407603 11533 ts_tablet_manager.cc:1905] T 4db557a52598476b957f53076628eb70 P 89b899be3f3b4dec819fdc0e2230e35d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:34.408219 11532 tablet_replica.cc:330] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c: stopping tablet replica
I20260811 00:12:34.408463 11532 raft_consensus.cc:2229] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:12:34.409041 11534 ts_tablet_manager.cc:1905] T 4db557a52598476b957f53076628eb70 P 18642798f024415cba62deab3184f14b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:34.409130 11532 raft_consensus.cc:2258] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:34.410063 11532 ts_tablet_manager.cc:1905] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:34.414678 11533 ts_tablet_manager.cc:1918] T 4db557a52598476b957f53076628eb70 P 89b899be3f3b4dec819fdc0e2230e35d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 00:12:34.414885 11533 log.cc:1196] T 4db557a52598476b957f53076628eb70 P 89b899be3f3b4dec819fdc0e2230e35d: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/4db557a52598476b957f53076628eb70
I20260811 00:12:34.415409 11533 ts_tablet_manager.cc:1939] T 4db557a52598476b957f53076628eb70 P 89b899be3f3b4dec819fdc0e2230e35d: Deleting consensus metadata
I20260811 00:12:34.416177 11534 ts_tablet_manager.cc:1918] T 4db557a52598476b957f53076628eb70 P 18642798f024415cba62deab3184f14b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 00:12:34.416380 11534 log.cc:1196] T 4db557a52598476b957f53076628eb70 P 18642798f024415cba62deab3184f14b: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/4db557a52598476b957f53076628eb70
I20260811 00:12:34.416780 10501 catalog_manager.cc:4841] TS 89b899be3f3b4dec819fdc0e2230e35d (127.7.94.3:44257): tablet 4db557a52598476b957f53076628eb70 (table test-table [id=a702fb18fc6b481aa437e8df36cc9329]) successfully deleted
I20260811 00:12:34.416880 11532 ts_tablet_manager.cc:1918] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 00:12:34.416939 11534 ts_tablet_manager.cc:1939] T 4db557a52598476b957f53076628eb70 P 18642798f024415cba62deab3184f14b: Deleting consensus metadata
I20260811 00:12:34.417007 11532 log.cc:1196] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/4db557a52598476b957f53076628eb70
I20260811 00:12:34.417516 11532 ts_tablet_manager.cc:1939] T 4db557a52598476b957f53076628eb70 P bd9b2df5cede4dc19d32067269d7e50c: Deleting consensus metadata
I20260811 00:12:34.418211 10498 catalog_manager.cc:4841] TS 18642798f024415cba62deab3184f14b (127.7.94.1:37555): tablet 4db557a52598476b957f53076628eb70 (table test-table [id=a702fb18fc6b481aa437e8df36cc9329]) successfully deleted
I20260811 00:12:34.418540 10501 catalog_manager.cc:4841] TS bd9b2df5cede4dc19d32067269d7e50c (127.7.94.2:34595): tablet 4db557a52598476b957f53076628eb70 (table test-table [id=a702fb18fc6b481aa437e8df36cc9329]) successfully deleted
Aug 11 00:12:34 dist-test-slave-8hhm krb5kdc[10454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407154, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 00:12:34 dist-test-slave-8hhm krb5kdc[10454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407154, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:34.457155 11547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:34.443801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48814 (local address 127.7.94.62:33893)
0811 00:12:34.444383 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:34.444387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:34.444418 (+ 31us) server_negotiation.cc:407] Connection header received
0811 00:12:34.444531 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:34.444535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:34.444609 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:34.444699 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:34.446022 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:34.446794 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:34.447961 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:34.448326 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:34.448410 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:34.452066 (+ 3656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:34.452071 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:34.452074 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:34.452106 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:34.454893 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:34.455487 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:34.455490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:34.455492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:34.455620 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:34.455909 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:34.455912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:34.455914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:34.456190 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:34.456361 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:34.456649 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 00:12:34.456810 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":290,"threads_started":1}
I20260811 00:12:34.460338 10529 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48814:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:12:34.460891 10529 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:12:34.467085 10751 tablet_service.cc:1467] Processing CreateTablet for tablet a89e30f111db4682b5dada25a3b4534d (DEFAULT_TABLE table=test-table [id=323c2fa031b84bbfb8adf7899ac04cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:34.467124 10991 tablet_service.cc:1467] Processing CreateTablet for tablet a89e30f111db4682b5dada25a3b4534d (DEFAULT_TABLE table=test-table [id=323c2fa031b84bbfb8adf7899ac04cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:34.467576 10751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a89e30f111db4682b5dada25a3b4534d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:34.467465 11228 tablet_service.cc:1467] Processing CreateTablet for tablet a89e30f111db4682b5dada25a3b4534d (DEFAULT_TABLE table=test-table [id=323c2fa031b84bbfb8adf7899ac04cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:34.467576 10991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a89e30f111db4682b5dada25a3b4534d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:34.467959 11228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a89e30f111db4682b5dada25a3b4534d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:34.472904 11549 tablet_bootstrap.cc:492] T a89e30f111db4682b5dada25a3b4534d P 89b899be3f3b4dec819fdc0e2230e35d: Bootstrap starting.
I20260811 00:12:34.474586 11552 tablet_bootstrap.cc:492] T a89e30f111db4682b5dada25a3b4534d P bd9b2df5cede4dc19d32067269d7e50c: Bootstrap starting.
I20260811 00:12:34.474833 11549 tablet_bootstrap.cc:654] T a89e30f111db4682b5dada25a3b4534d P 89b899be3f3b4dec819fdc0e2230e35d: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:34.475425 11551 tablet_bootstrap.cc:492] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b: Bootstrap starting.
I20260811 00:12:34.476881 11552 tablet_bootstrap.cc:654] T a89e30f111db4682b5dada25a3b4534d P bd9b2df5cede4dc19d32067269d7e50c: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:34.476974 11551 tablet_bootstrap.cc:654] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:34.479143 11551 tablet_bootstrap.cc:492] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b: No bootstrap required, opened a new log
I20260811 00:12:34.479280 11551 ts_tablet_manager.cc:1397] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 00:12:34.479336 11552 tablet_bootstrap.cc:492] T a89e30f111db4682b5dada25a3b4534d P bd9b2df5cede4dc19d32067269d7e50c: No bootstrap required, opened a new log
I20260811 00:12:34.479462 11552 ts_tablet_manager.cc:1397] T a89e30f111db4682b5dada25a3b4534d P bd9b2df5cede4dc19d32067269d7e50c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 00:12:34.479610 11549 tablet_bootstrap.cc:492] T a89e30f111db4682b5dada25a3b4534d P 89b899be3f3b4dec819fdc0e2230e35d: No bootstrap required, opened a new log
I20260811 00:12:34.479733 11549 ts_tablet_manager.cc:1397] T a89e30f111db4682b5dada25a3b4534d P 89b899be3f3b4dec819fdc0e2230e35d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 00:12:34.480047 11551 raft_consensus.cc:348] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:34.480286 11551 raft_consensus.cc:374] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:34.480382 11551 raft_consensus.cc:729] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18642798f024415cba62deab3184f14b, State: Initialized, Role: FOLLOWER
I20260811 00:12:34.480242 11552 raft_consensus.cc:348] T a89e30f111db4682b5dada25a3b4534d P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:34.480471 11552 raft_consensus.cc:374] T a89e30f111db4682b5dada25a3b4534d P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:34.480453 11549 raft_consensus.cc:348] T a89e30f111db4682b5dada25a3b4534d P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:34.480638 11552 raft_consensus.cc:729] T a89e30f111db4682b5dada25a3b4534d P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd9b2df5cede4dc19d32067269d7e50c, State: Initialized, Role: FOLLOWER
I20260811 00:12:34.480775 11549 raft_consensus.cc:374] T a89e30f111db4682b5dada25a3b4534d P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:34.480638 11551 consensus_queue.cc:260] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:34.480880 11549 raft_consensus.cc:729] T a89e30f111db4682b5dada25a3b4534d P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89b899be3f3b4dec819fdc0e2230e35d, State: Initialized, Role: FOLLOWER
I20260811 00:12:34.480908 11552 consensus_queue.cc:260] T a89e30f111db4682b5dada25a3b4534d P bd9b2df5cede4dc19d32067269d7e50c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:34.481117 11549 consensus_queue.cc:260] T a89e30f111db4682b5dada25a3b4534d P 89b899be3f3b4dec819fdc0e2230e35d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:34.481606 11551 ts_tablet_manager.cc:1428] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 00:12:34.481690 11549 ts_tablet_manager.cc:1428] T a89e30f111db4682b5dada25a3b4534d P 89b899be3f3b4dec819fdc0e2230e35d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 00:12:34.482224 11552 ts_tablet_manager.cc:1428] T a89e30f111db4682b5dada25a3b4534d P bd9b2df5cede4dc19d32067269d7e50c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 00:12:34.558331 11557 raft_consensus.cc:482] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:12:34.558570 11557 raft_consensus.cc:504] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:34.559433 11557 leader_election.cc:288] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd9b2df5cede4dc19d32067269d7e50c (127.7.94.2:34595), 89b899be3f3b4dec819fdc0e2230e35d (127.7.94.3:44257)
I20260811 00:12:34.560078 11019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a89e30f111db4682b5dada25a3b4534d" candidate_uuid: "18642798f024415cba62deab3184f14b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd9b2df5cede4dc19d32067269d7e50c" is_pre_election: true
I20260811 00:12:34.560101 11257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a89e30f111db4682b5dada25a3b4534d" candidate_uuid: "18642798f024415cba62deab3184f14b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89b899be3f3b4dec819fdc0e2230e35d" is_pre_election: true
I20260811 00:12:34.560382 11019 raft_consensus.cc:2454] T a89e30f111db4682b5dada25a3b4534d P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18642798f024415cba62deab3184f14b in term 0.
I20260811 00:12:34.560410 11257 raft_consensus.cc:2454] T a89e30f111db4682b5dada25a3b4534d P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18642798f024415cba62deab3184f14b in term 0.
I20260811 00:12:34.560890 10673 leader_election.cc:302] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18642798f024415cba62deab3184f14b, 89b899be3f3b4dec819fdc0e2230e35d; no voters:
I20260811 00:12:34.561136 11557 raft_consensus.cc:2781] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:12:34.561280 11557 raft_consensus.cc:482] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:12:34.561378 11557 raft_consensus.cc:3037] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:34.563386 11557 raft_consensus.cc:504] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:34.563910 11557 leader_election.cc:288] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [CANDIDATE]: Term 1 election: Requested vote from peers bd9b2df5cede4dc19d32067269d7e50c (127.7.94.2:34595), 89b899be3f3b4dec819fdc0e2230e35d (127.7.94.3:44257)
I20260811 00:12:34.564291 11019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a89e30f111db4682b5dada25a3b4534d" candidate_uuid: "18642798f024415cba62deab3184f14b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd9b2df5cede4dc19d32067269d7e50c"
I20260811 00:12:34.564525 11019 raft_consensus.cc:3037] T a89e30f111db4682b5dada25a3b4534d P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:34.564460 11257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a89e30f111db4682b5dada25a3b4534d" candidate_uuid: "18642798f024415cba62deab3184f14b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89b899be3f3b4dec819fdc0e2230e35d"
I20260811 00:12:34.564677 11257 raft_consensus.cc:3037] T a89e30f111db4682b5dada25a3b4534d P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:34.566712 11019 raft_consensus.cc:2454] T a89e30f111db4682b5dada25a3b4534d P bd9b2df5cede4dc19d32067269d7e50c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18642798f024415cba62deab3184f14b in term 1.
I20260811 00:12:34.566715 11257 raft_consensus.cc:2454] T a89e30f111db4682b5dada25a3b4534d P 89b899be3f3b4dec819fdc0e2230e35d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18642798f024415cba62deab3184f14b in term 1.
I20260811 00:12:34.567077 10673 leader_election.cc:302] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18642798f024415cba62deab3184f14b, 89b899be3f3b4dec819fdc0e2230e35d; no voters:
I20260811 00:12:34.567350 11557 raft_consensus.cc:2781] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:34.567855 11557 raft_consensus.cc:686] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [term 1 LEADER]: Becoming Leader. State: Replica: 18642798f024415cba62deab3184f14b, State: Running, Role: LEADER
I20260811 00:12:34.568185 11557 consensus_queue.cc:237] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:34.570881 10520 catalog_manager.cc:5495] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b reported cstate change: term changed from 0 to 1, leader changed from <none> to 18642798f024415cba62deab3184f14b (127.7.94.1). New cstate: current_term: 1 leader_uuid: "18642798f024415cba62deab3184f14b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } health_report { overall_health: UNKNOWN } } }
I20260811 00:12:34.611603 11563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:34.606742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36360 (local address 127.7.94.1:37555)
0811 00:12:34.607396 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:34.607400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:34.607419 (+ 19us) server_negotiation.cc:407] Connection header received
0811 00:12:34.607472 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:34.607475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:34.607549 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:34.607642 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:12:34.607962 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:34.608736 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:34.609975 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:34.610389 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:34.610484 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:34.610839 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:12:34.610956 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:12:34.611025 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:34.611144 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 00:12:34.611206 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":364,"threads_started":1}
I20260811 00:12:34.620765 11257 raft_consensus.cc:1261] T a89e30f111db4682b5dada25a3b4534d P 89b899be3f3b4dec819fdc0e2230e35d [term 1 FOLLOWER]: Refusing update from remote peer 18642798f024415cba62deab3184f14b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:12:34.620815 11019 raft_consensus.cc:1261] T a89e30f111db4682b5dada25a3b4534d P bd9b2df5cede4dc19d32067269d7e50c [term 1 FOLLOWER]: Refusing update from remote peer 18642798f024415cba62deab3184f14b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:12:34.621371 11560 consensus_queue.cc:1035] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [LEADER]: Connected to new peer: Peer: permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:12:34.622110 11557 consensus_queue.cc:1035] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:12:34.628537 11504 mvcc.cc:205] Tried to move back new op lower bound from 7317123705321517056 to 7317123705113890816. Current Snapshot: MvccSnapshot[applied={T|T < 7317123705321517056}]
I20260811 00:12:34.677062 10520 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:12:34.677381 10520 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:12:34.680763 10520 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc: Sending DeleteTablet for 3 replicas of tablet a89e30f111db4682b5dada25a3b4534d
I20260811 00:12:34.681783 10991 tablet_service.cc:1514] Processing DeleteTablet for tablet a89e30f111db4682b5dada25a3b4534d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:34 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:48608
I20260811 00:12:34.681870 11228 tablet_service.cc:1514] Processing DeleteTablet for tablet a89e30f111db4682b5dada25a3b4534d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:34 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:44082
I20260811 00:12:34.682121 11532 tablet_replica.cc:330] T a89e30f111db4682b5dada25a3b4534d P bd9b2df5cede4dc19d32067269d7e50c: stopping tablet replica
I20260811 00:12:34.682145 10751 tablet_service.cc:1514] Processing DeleteTablet for tablet a89e30f111db4682b5dada25a3b4534d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:34 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:36346
I20260811 00:12:34.682359 11532 raft_consensus.cc:2229] T a89e30f111db4682b5dada25a3b4534d P bd9b2df5cede4dc19d32067269d7e50c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:12:34.682427 11534 tablet_replica.cc:330] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b: stopping tablet replica
I20260811 00:12:34.682474 11533 tablet_replica.cc:330] T a89e30f111db4682b5dada25a3b4534d P 89b899be3f3b4dec819fdc0e2230e35d: stopping tablet replica
I20260811 00:12:34.682572 11532 raft_consensus.cc:2258] T a89e30f111db4682b5dada25a3b4534d P bd9b2df5cede4dc19d32067269d7e50c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:34.682672 11534 raft_consensus.cc:2229] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:12:34.682686 11533 raft_consensus.cc:2229] T a89e30f111db4682b5dada25a3b4534d P 89b899be3f3b4dec819fdc0e2230e35d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:12:34.682859 11533 raft_consensus.cc:2258] T a89e30f111db4682b5dada25a3b4534d P 89b899be3f3b4dec819fdc0e2230e35d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:34.683002 11534 raft_consensus.cc:2258] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:34.683919 11533 ts_tablet_manager.cc:1905] T a89e30f111db4682b5dada25a3b4534d P 89b899be3f3b4dec819fdc0e2230e35d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:34.684266 11534 ts_tablet_manager.cc:1905] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:34.686087 10520 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48814:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:12:34.686896 10520 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:12:34.686975 11532 ts_tablet_manager.cc:1905] T a89e30f111db4682b5dada25a3b4534d P bd9b2df5cede4dc19d32067269d7e50c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:34.693976 10991 tablet_service.cc:1467] Processing CreateTablet for tablet 173097717a69484c86fd08f2e359a9a8 (DEFAULT_TABLE table=test-table [id=bbc64c575efc407aa4c02a92daf0c4ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:34.694005 10751 tablet_service.cc:1467] Processing CreateTablet for tablet 173097717a69484c86fd08f2e359a9a8 (DEFAULT_TABLE table=test-table [id=bbc64c575efc407aa4c02a92daf0c4ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:34.694190 11228 tablet_service.cc:1467] Processing CreateTablet for tablet 173097717a69484c86fd08f2e359a9a8 (DEFAULT_TABLE table=test-table [id=bbc64c575efc407aa4c02a92daf0c4ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:34.734953 10991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 173097717a69484c86fd08f2e359a9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:34.735296 11228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 173097717a69484c86fd08f2e359a9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:34.735515 10751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 173097717a69484c86fd08f2e359a9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:34.742468 11551 tablet_bootstrap.cc:492] T 173097717a69484c86fd08f2e359a9a8 P 18642798f024415cba62deab3184f14b: Bootstrap starting.
I20260811 00:12:34.743052 11549 tablet_bootstrap.cc:492] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d: Bootstrap starting.
I20260811 00:12:34.744283 11551 tablet_bootstrap.cc:654] T 173097717a69484c86fd08f2e359a9a8 P 18642798f024415cba62deab3184f14b: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:34.744735 11549 tablet_bootstrap.cc:654] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:34.745533 11552 tablet_bootstrap.cc:492] T 173097717a69484c86fd08f2e359a9a8 P bd9b2df5cede4dc19d32067269d7e50c: Bootstrap starting.
I20260811 00:12:34.747352 11552 tablet_bootstrap.cc:654] T 173097717a69484c86fd08f2e359a9a8 P bd9b2df5cede4dc19d32067269d7e50c: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:34.747833 11549 tablet_bootstrap.cc:492] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d: No bootstrap required, opened a new log
I20260811 00:12:34.747833 11551 tablet_bootstrap.cc:492] T 173097717a69484c86fd08f2e359a9a8 P 18642798f024415cba62deab3184f14b: No bootstrap required, opened a new log
I20260811 00:12:34.748018 11551 ts_tablet_manager.cc:1397] T 173097717a69484c86fd08f2e359a9a8 P 18642798f024415cba62deab3184f14b: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260811 00:12:34.748003 11549 ts_tablet_manager.cc:1397] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 00:12:34.748754 11551 raft_consensus.cc:348] T 173097717a69484c86fd08f2e359a9a8 P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:34.748782 11549 raft_consensus.cc:348] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:34.749055 11549 raft_consensus.cc:374] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:34.749045 11551 raft_consensus.cc:374] T 173097717a69484c86fd08f2e359a9a8 P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:34.749148 11549 raft_consensus.cc:729] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89b899be3f3b4dec819fdc0e2230e35d, State: Initialized, Role: FOLLOWER
I20260811 00:12:34.749148 11551 raft_consensus.cc:729] T 173097717a69484c86fd08f2e359a9a8 P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18642798f024415cba62deab3184f14b, State: Initialized, Role: FOLLOWER
I20260811 00:12:34.749361 11552 tablet_bootstrap.cc:492] T 173097717a69484c86fd08f2e359a9a8 P bd9b2df5cede4dc19d32067269d7e50c: No bootstrap required, opened a new log
I20260811 00:12:34.749527 11552 ts_tablet_manager.cc:1397] T 173097717a69484c86fd08f2e359a9a8 P bd9b2df5cede4dc19d32067269d7e50c: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260811 00:12:34.749415 11551 consensus_queue.cc:260] T 173097717a69484c86fd08f2e359a9a8 P 18642798f024415cba62deab3184f14b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:34.749415 11549 consensus_queue.cc:260] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:34.749927 11549 ts_tablet_manager.cc:1428] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 00:12:34.749949 11551 ts_tablet_manager.cc:1428] T 173097717a69484c86fd08f2e359a9a8 P 18642798f024415cba62deab3184f14b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 00:12:34.750193 11552 raft_consensus.cc:348] T 173097717a69484c86fd08f2e359a9a8 P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:34.750437 11552 raft_consensus.cc:374] T 173097717a69484c86fd08f2e359a9a8 P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:34.750538 11552 raft_consensus.cc:729] T 173097717a69484c86fd08f2e359a9a8 P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd9b2df5cede4dc19d32067269d7e50c, State: Initialized, Role: FOLLOWER
I20260811 00:12:34.750880 11552 consensus_queue.cc:260] T 173097717a69484c86fd08f2e359a9a8 P bd9b2df5cede4dc19d32067269d7e50c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:34.752135 11552 ts_tablet_manager.cc:1428] T 173097717a69484c86fd08f2e359a9a8 P bd9b2df5cede4dc19d32067269d7e50c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 00:12:34.757661 11534 ts_tablet_manager.cc:1918] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:12:34.757838 11534 log.cc:1196] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/a89e30f111db4682b5dada25a3b4534d
I20260811 00:12:34.758266 11534 ts_tablet_manager.cc:1939] T a89e30f111db4682b5dada25a3b4534d P 18642798f024415cba62deab3184f14b: Deleting consensus metadata
I20260811 00:12:34.758773 11532 ts_tablet_manager.cc:1918] T a89e30f111db4682b5dada25a3b4534d P bd9b2df5cede4dc19d32067269d7e50c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:12:34.758934 11532 log.cc:1196] T a89e30f111db4682b5dada25a3b4534d P bd9b2df5cede4dc19d32067269d7e50c: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/a89e30f111db4682b5dada25a3b4534d
I20260811 00:12:34.759392 11532 ts_tablet_manager.cc:1939] T a89e30f111db4682b5dada25a3b4534d P bd9b2df5cede4dc19d32067269d7e50c: Deleting consensus metadata
I20260811 00:12:34.759827 11533 ts_tablet_manager.cc:1918] T a89e30f111db4682b5dada25a3b4534d P 89b899be3f3b4dec819fdc0e2230e35d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:12:34.760005 11533 log.cc:1196] T a89e30f111db4682b5dada25a3b4534d P 89b899be3f3b4dec819fdc0e2230e35d: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/a89e30f111db4682b5dada25a3b4534d
I20260811 00:12:34.760715 11533 ts_tablet_manager.cc:1939] T a89e30f111db4682b5dada25a3b4534d P 89b899be3f3b4dec819fdc0e2230e35d: Deleting consensus metadata
I20260811 00:12:34.792975 10498 catalog_manager.cc:4841] TS 18642798f024415cba62deab3184f14b (127.7.94.1:37555): tablet a89e30f111db4682b5dada25a3b4534d (table test-table [id=323c2fa031b84bbfb8adf7899ac04cf1]) successfully deleted
I20260811 00:12:34.813175 10501 catalog_manager.cc:4841] TS bd9b2df5cede4dc19d32067269d7e50c (127.7.94.2:34595): tablet a89e30f111db4682b5dada25a3b4534d (table test-table [id=323c2fa031b84bbfb8adf7899ac04cf1]) successfully deleted
I20260811 00:12:34.838514 10501 catalog_manager.cc:4841] TS 89b899be3f3b4dec819fdc0e2230e35d (127.7.94.3:44257): tablet a89e30f111db4682b5dada25a3b4534d (table test-table [id=323c2fa031b84bbfb8adf7899ac04cf1]) successfully deleted
I20260811 00:12:34.915126 11502 raft_consensus.cc:482] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:12:34.915409 11502 raft_consensus.cc:504] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:34.916131 11502 leader_election.cc:288] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 18642798f024415cba62deab3184f14b (127.7.94.1:37555), bd9b2df5cede4dc19d32067269d7e50c (127.7.94.2:34595)
I20260811 00:12:34.916841 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "173097717a69484c86fd08f2e359a9a8" candidate_uuid: "89b899be3f3b4dec819fdc0e2230e35d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd9b2df5cede4dc19d32067269d7e50c" is_pre_election: true
I20260811 00:12:34.916853 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "173097717a69484c86fd08f2e359a9a8" candidate_uuid: "89b899be3f3b4dec819fdc0e2230e35d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18642798f024415cba62deab3184f14b" is_pre_election: true
I20260811 00:12:34.917114 11017 raft_consensus.cc:2454] T 173097717a69484c86fd08f2e359a9a8 P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89b899be3f3b4dec819fdc0e2230e35d in term 0.
I20260811 00:12:34.917142 10794 raft_consensus.cc:2454] T 173097717a69484c86fd08f2e359a9a8 P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89b899be3f3b4dec819fdc0e2230e35d in term 0.
I20260811 00:12:34.917635 11151 leader_election.cc:302] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89b899be3f3b4dec819fdc0e2230e35d, bd9b2df5cede4dc19d32067269d7e50c; no voters:
I20260811 00:12:34.918004 11502 raft_consensus.cc:2781] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:12:34.918141 11502 raft_consensus.cc:482] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:12:34.918238 11502 raft_consensus.cc:3037] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:34.953670 11502 raft_consensus.cc:504] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:34.954571 11502 leader_election.cc:288] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [CANDIDATE]: Term 1 election: Requested vote from peers 18642798f024415cba62deab3184f14b (127.7.94.1:37555), bd9b2df5cede4dc19d32067269d7e50c (127.7.94.2:34595)
I20260811 00:12:34.955225 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "173097717a69484c86fd08f2e359a9a8" candidate_uuid: "89b899be3f3b4dec819fdc0e2230e35d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd9b2df5cede4dc19d32067269d7e50c"
I20260811 00:12:34.955256 10794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "173097717a69484c86fd08f2e359a9a8" candidate_uuid: "89b899be3f3b4dec819fdc0e2230e35d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "18642798f024415cba62deab3184f14b"
I20260811 00:12:34.955464 11017 raft_consensus.cc:3037] T 173097717a69484c86fd08f2e359a9a8 P bd9b2df5cede4dc19d32067269d7e50c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:34.955538 10794 raft_consensus.cc:3037] T 173097717a69484c86fd08f2e359a9a8 P 18642798f024415cba62deab3184f14b [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:34.958127 11017 raft_consensus.cc:2454] T 173097717a69484c86fd08f2e359a9a8 P bd9b2df5cede4dc19d32067269d7e50c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89b899be3f3b4dec819fdc0e2230e35d in term 1.
I20260811 00:12:34.958600 11151 leader_election.cc:302] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89b899be3f3b4dec819fdc0e2230e35d, bd9b2df5cede4dc19d32067269d7e50c; no voters:
I20260811 00:12:34.958909 11502 raft_consensus.cc:2781] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:34.959070 11502 raft_consensus.cc:686] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [term 1 LEADER]: Becoming Leader. State: Replica: 89b899be3f3b4dec819fdc0e2230e35d, State: Running, Role: LEADER
I20260811 00:12:34.959339 11502 consensus_queue.cc:237] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } }
I20260811 00:12:34.961827 10526 catalog_manager.cc:5495] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d reported cstate change: term changed from 0 to 1, leader changed from <none> to 89b899be3f3b4dec819fdc0e2230e35d (127.7.94.3). New cstate: current_term: 1 leader_uuid: "89b899be3f3b4dec819fdc0e2230e35d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89b899be3f3b4dec819fdc0e2230e35d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44257 } health_report { overall_health: HEALTHY } } }
I20260811 00:12:35.007090 11576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:35.001374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44108 (local address 127.7.94.3:44257)
0811 00:12:35.002098 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:35.002103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:35.002121 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:12:35.002171 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:35.002174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:35.002232 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:35.002343 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:12:35.002774 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:35.003593 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:35.005025 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:35.005693 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:35.005832 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:35.006306 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:12:35.006413 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:12:35.006518 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:35.006679 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 00:12:35.006757 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":417,"threads_started":1}
I20260811 00:12:35.020661 11576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:35.015760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:58911 (local address 127.7.94.3:44257)
0811 00:12:35.015947 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:35.015953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:35.016393 (+ 440us) server_negotiation.cc:407] Connection header received
0811 00:12:35.016534 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:35.016537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:35.016597 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:35.016673 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:12:35.017088 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:35.017837 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:35.019107 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:35.019494 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:35.019585 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:35.020020 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:12:35.020135 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:12:35.020210 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:35.020331 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 00:12:35.020407 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":48}
I20260811 00:12:35.020713 11580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:35.015584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:44257 (local address 127.7.94.3:58911)
0811 00:12:35.016305 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:35.016325 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:35.016431 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:35.016700 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:35.016703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:35.016712 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:12:35.016959 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:35.016966 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:35.017955 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:35.017958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:35.018984 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:35.018995 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:35.019645 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:35.019649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:35.019773 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:35.019805 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:12:35.020226 (+ 421us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:12:35.020234 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 00:12:35.020294 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 00:12:35.020386 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":453,"threads_started":1}
I20260811 00:12:35.029428 11017 raft_consensus.cc:1261] T 173097717a69484c86fd08f2e359a9a8 P bd9b2df5cede4dc19d32067269d7e50c [term 1 FOLLOWER]: Refusing update from remote peer 89b899be3f3b4dec819fdc0e2230e35d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:12:35.029973 11417 consensus_queue.cc:1035] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd9b2df5cede4dc19d32067269d7e50c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 00:12:35.047624 11499 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 00:12:35.050135 10794 raft_consensus.cc:2454] T 173097717a69484c86fd08f2e359a9a8 P 18642798f024415cba62deab3184f14b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89b899be3f3b4dec819fdc0e2230e35d in term 1.
I20260811 00:12:35.050518 10789 raft_consensus.cc:1261] T 173097717a69484c86fd08f2e359a9a8 P 18642798f024415cba62deab3184f14b [term 1 FOLLOWER]: Refusing update from remote peer 89b899be3f3b4dec819fdc0e2230e35d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:12:35.051491 11518 consensus_queue.cc:1035] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [LEADER]: Connected to new peer: Peer: permanent_uuid: "18642798f024415cba62deab3184f14b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:12:36.639272 10527 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:12:36.639576 10527 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:12:36.643864 10527 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a158352fd6fb49d78036e80cc18fcbdc: Sending DeleteTablet for 3 replicas of tablet 173097717a69484c86fd08f2e359a9a8
I20260811 00:12:36.644659 10751 tablet_service.cc:1514] Processing DeleteTablet for tablet 173097717a69484c86fd08f2e359a9a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:36 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:36346
I20260811 00:12:36.644739 11228 tablet_service.cc:1514] Processing DeleteTablet for tablet 173097717a69484c86fd08f2e359a9a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:36 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:44082
I20260811 00:12:36.644798 10991 tablet_service.cc:1514] Processing DeleteTablet for tablet 173097717a69484c86fd08f2e359a9a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:36 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:48608
I20260811 00:12:36.645428 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 10641
I20260811 00:12:36.657469 11626 tablet_replica.cc:330] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d: stopping tablet replica
I20260811 00:12:36.657801 11626 raft_consensus.cc:2229] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:12:36.658526 11626 raft_consensus.cc:2258] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:36.659865 11625 tablet_replica.cc:330] T 173097717a69484c86fd08f2e359a9a8 P 18642798f024415cba62deab3184f14b: stopping tablet replica
I20260811 00:12:36.660064 11625 raft_consensus.cc:2229] T 173097717a69484c86fd08f2e359a9a8 P 18642798f024415cba62deab3184f14b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:12:36.660323 11626 ts_tablet_manager.cc:1905] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:36.660323 11625 raft_consensus.cc:2258] T 173097717a69484c86fd08f2e359a9a8 P 18642798f024415cba62deab3184f14b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:36.660964 11627 tablet_replica.cc:330] T 173097717a69484c86fd08f2e359a9a8 P bd9b2df5cede4dc19d32067269d7e50c: stopping tablet replica
I20260811 00:12:36.661172 11627 raft_consensus.cc:2229] T 173097717a69484c86fd08f2e359a9a8 P bd9b2df5cede4dc19d32067269d7e50c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:12:36.661449 11625 ts_tablet_manager.cc:1905] T 173097717a69484c86fd08f2e359a9a8 P 18642798f024415cba62deab3184f14b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:36.661448 11627 raft_consensus.cc:2258] T 173097717a69484c86fd08f2e359a9a8 P bd9b2df5cede4dc19d32067269d7e50c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:36.662372 11627 ts_tablet_manager.cc:1905] T 173097717a69484c86fd08f2e359a9a8 P bd9b2df5cede4dc19d32067269d7e50c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:36.667585 11626 ts_tablet_manager.cc:1918] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 00:12:36.667732 11626 log.cc:1196] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/173097717a69484c86fd08f2e359a9a8
I20260811 00:12:36.668215 11626 ts_tablet_manager.cc:1939] T 173097717a69484c86fd08f2e359a9a8 P 89b899be3f3b4dec819fdc0e2230e35d: Deleting consensus metadata
I20260811 00:12:36.668445 11625 ts_tablet_manager.cc:1918] T 173097717a69484c86fd08f2e359a9a8 P 18642798f024415cba62deab3184f14b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 00:12:36.668581 11625 log.cc:1196] T 173097717a69484c86fd08f2e359a9a8 P 18642798f024415cba62deab3184f14b: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/173097717a69484c86fd08f2e359a9a8
I20260811 00:12:36.668969 11625 ts_tablet_manager.cc:1939] T 173097717a69484c86fd08f2e359a9a8 P 18642798f024415cba62deab3184f14b: Deleting consensus metadata
I20260811 00:12:36.668956 11627 ts_tablet_manager.cc:1918] T 173097717a69484c86fd08f2e359a9a8 P bd9b2df5cede4dc19d32067269d7e50c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 00:12:36.669099 11627 log.cc:1196] T 173097717a69484c86fd08f2e359a9a8 P bd9b2df5cede4dc19d32067269d7e50c: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/173097717a69484c86fd08f2e359a9a8
I20260811 00:12:36.669449 10501 catalog_manager.cc:4841] TS 89b899be3f3b4dec819fdc0e2230e35d (127.7.94.3:44257): tablet 173097717a69484c86fd08f2e359a9a8 (table test-table [id=bbc64c575efc407aa4c02a92daf0c4ae]) successfully deleted
I20260811 00:12:36.669540 11627 ts_tablet_manager.cc:1939] T 173097717a69484c86fd08f2e359a9a8 P bd9b2df5cede4dc19d32067269d7e50c: Deleting consensus metadata
I20260811 00:12:36.670070 10498 catalog_manager.cc:4841] TS 18642798f024415cba62deab3184f14b (127.7.94.1:37555): tablet 173097717a69484c86fd08f2e359a9a8 (table test-table [id=bbc64c575efc407aa4c02a92daf0c4ae]) successfully deleted
I20260811 00:12:36.670444 10501 catalog_manager.cc:4841] TS bd9b2df5cede4dc19d32067269d7e50c (127.7.94.2:34595): tablet 173097717a69484c86fd08f2e359a9a8 (table test-table [id=bbc64c575efc407aa4c02a92daf0c4ae]) successfully deleted
Aug 11 00:12:36 dist-test-slave-8hhm krb5kdc[10454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407154, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:12:36.680830 11629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:36.649073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36376 (local address 127.7.94.1:37555)
0811 00:12:36.654171 (+ 5098us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:36.654181 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:36.660173 (+ 5992us) server_negotiation.cc:407] Connection header received
0811 00:12:36.660596 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:36.660605 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:36.660758 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:36.660844 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:36.662585 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:36.663751 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:36.665549 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:36.666260 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:36.666362 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:36.675663 (+ 9301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:36.675678 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:36.675694 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:36.675733 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:36.678490 (+ 2757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:36.679146 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:36.679153 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:36.679155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:36.679291 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:36.679613 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:36.679617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:36.679618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:36.679952 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:36.680161 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:36.680389 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 00:12:36.680566 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4890,"thread_start_us":4709,"threads_started":1}
W20260811 00:12:36.681470 10863 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36376
W20260811 00:12:36.682046 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 10641: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:12:36.682225 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 10641
I20260811 00:12:36.713985 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 10877
Aug 11 00:12:36 dist-test-slave-8hhm krb5kdc[10454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407154, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.94.2@KRBTEST.COM
I20260811 00:12:36.729595 11634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:36.714812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48622 (local address 127.7.94.2:34595)
0811 00:12:36.715425 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:36.715429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:36.715445 (+ 16us) server_negotiation.cc:407] Connection header received
0811 00:12:36.715509 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:36.715512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:36.715572 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:36.715687 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:36.716940 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:36.717770 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:36.718953 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:36.719317 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:36.719412 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:36.723496 (+ 4084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:36.723511 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:36.723526 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:36.723562 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:36.726795 (+ 3233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:36.727583 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:36.727591 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:36.727593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:36.727739 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:36.728098 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:36.728101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:36.728103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:36.728441 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:36.728668 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:36.728953 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 00:12:36.729107 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":328,"threads_started":1}
W20260811 00:12:36.730384 11110 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48622
W20260811 00:12:36.731038 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 10877: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:12:36.731199 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 10877
I20260811 00:12:36.762465 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 11125
Aug 11 00:12:36 dist-test-slave-8hhm krb5kdc[10454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407154, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.94.3@KRBTEST.COM
I20260811 00:12:36.779431 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:36.763372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44124 (local address 127.7.94.3:44257)
0811 00:12:36.763983 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:36.763988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:36.764009 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:12:36.764079 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:36.764083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:36.764154 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:36.764257 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:36.765626 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:36.766478 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:36.767696 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:36.768201 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:36.768306 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:36.772838 (+ 4532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:36.772854 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:36.772884 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:36.772928 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:36.776273 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:36.777020 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:36.777029 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:36.777031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:36.777171 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:36.777644 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:36.777671 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:36.777674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:36.778111 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:36.778454 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:36.778783 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 00:12:36.779036 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":330,"threads_started":1}
W20260811 00:12:36.780189 11342 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44124
W20260811 00:12:36.780854 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 11125: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:12:36.781056 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 11125
I20260811 00:12:36.817094 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 10473
I20260811 00:12:36.818461 10589 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:36.939160 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 10473
2026-08-11T00:12:36Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8791 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:12:37 dist-test-slave-8hhm krb5kdc[11643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:12:37 dist-test-slave-8hhm krb5kdc[11643](info): set up 2 sockets
Aug 11 00:12:37 dist-test-slave-8hhm krb5kdc[11643](info): commencing operation
krb5kdc: starting...
W20260811 00:12:38.262657 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.003s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:12:38 dist-test-slave-8hhm krb5kdc[11643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:12:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:12:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:12:38.476662 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:44573
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:34023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:44573
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:38.805056 11659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:38.805465 11659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:38.805683 11659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:38.813040 11659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:12:38.813167 11659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:38.813285 11659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:38.813390 11659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:12:38.813473 11659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:12:38.823326 11659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:34023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:44573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:44573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:38.824740 11659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:38.826112 11659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:38.838164 11669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:38.838135 11666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:38.838220 11667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:38.838572 11659 server_base.cc:1034] running on GCE node
I20260811 00:12:38.839561 11659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:38.841012 11659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:38.842253 11659 hybrid_clock.cc:648] HybridClock initialized: now 1786407158842218 us; error 47 us; skew 500 ppm
I20260811 00:12:38.842600 11659 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:38 dist-test-slave-8hhm krb5kdc[11643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407158, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:38.846071 11659 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:12:38.847503 11659 webserver.cc:460] Webserver started at http://127.7.94.62:38177/ using document root <none> and password file <none>
I20260811 00:12:38.848160 11659 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:38.848258 11659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:38.848517 11659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:38.851513 11659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "731b82d6054548989452ec6774c8b6dd"
format_stamp: "Formatted at 2026-08-11 00:12:38 on dist-test-slave-8hhm"
server_key: "3d865c578f09d9cdd82d6e1626695fb5"
server_key_iv: "8140defcd4934ba805941f3a0743327c"
server_key_version: "encryptionkey@0"
I20260811 00:12:38.852275 11659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "731b82d6054548989452ec6774c8b6dd"
format_stamp: "Formatted at 2026-08-11 00:12:38 on dist-test-slave-8hhm"
server_key: "3d865c578f09d9cdd82d6e1626695fb5"
server_key_iv: "8140defcd4934ba805941f3a0743327c"
server_key_version: "encryptionkey@0"
I20260811 00:12:38.857435 11659 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260811 00:12:38.860677 11681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:38.862015 11659 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:12:38.862221 11659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "731b82d6054548989452ec6774c8b6dd"
format_stamp: "Formatted at 2026-08-11 00:12:38 on dist-test-slave-8hhm"
server_key: "3d865c578f09d9cdd82d6e1626695fb5"
server_key_iv: "8140defcd4934ba805941f3a0743327c"
server_key_version: "encryptionkey@0"
I20260811 00:12:38.862411 11659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:38.882553 11659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:38.888394 11659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:38.888767 11659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:38.906332 11659 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:44573
I20260811 00:12:38.906361 11761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:44573 every 8 connection(s)
I20260811 00:12:38.907701 11659 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:12:38.911507 11763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:38.917546 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 11659
I20260811 00:12:38.917801 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:12:38.918630 7544 external_mini_cluster.cc:1428] Setting key 17ac767da523f3e7f207443c0c43759f
I20260811 00:12:38.922899 11763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd: Bootstrap starting.
I20260811 00:12:38.926646 11763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:38.927839 11763 log.cc:824] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:38.930924 11763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd: No bootstrap required, opened a new log
Aug 11 00:12:38 dist-test-slave-8hhm krb5kdc[11643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:38.938985 11763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "731b82d6054548989452ec6774c8b6dd" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44573 } }
I20260811 00:12:38.939343 11763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:38.939466 11763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 731b82d6054548989452ec6774c8b6dd, State: Initialized, Role: FOLLOWER
I20260811 00:12:38.940171 11763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "731b82d6054548989452ec6774c8b6dd" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44573 } }
I20260811 00:12:38.940474 11763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:12:38.940649 11763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:12:38.940866 11763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:38.941730 11771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:38.920700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39086 (local address 127.7.94.62:44573)
0811 00:12:38.921518 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:38.921529 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:38.921569 (+ 40us) server_negotiation.cc:407] Connection header received
0811 00:12:38.923331 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:38.923354 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:38.923731 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:38.924039 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:38.925927 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:38.927255 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:38.928895 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:38.929600 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:38.929748 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:38.934335 (+ 4587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:38.934342 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:38.934355 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:38.934391 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:38.937602 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:38.938311 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:38.938321 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:38.938333 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:38.938507 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:38.938877 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:38.938882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:38.938887 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:38.939498 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:38.939777 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:38.940169 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 00:12:38.940524 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":383,"threads_started":1}
I20260811 00:12:38.943404 11763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "731b82d6054548989452ec6774c8b6dd" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44573 } }
I20260811 00:12:38.943915 11763 leader_election.cc:302] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 731b82d6054548989452ec6774c8b6dd; no voters:
I20260811 00:12:38.944592 11763 leader_election.cc:288] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:12:38.944700 11773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:38.945426 11773 raft_consensus.cc:686] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd [term 1 LEADER]: Becoming Leader. State: Replica: 731b82d6054548989452ec6774c8b6dd, State: Running, Role: LEADER
I20260811 00:12:38.946007 11773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "731b82d6054548989452ec6774c8b6dd" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44573 } }
I20260811 00:12:38.946882 11763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:12:38.948814 11776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "731b82d6054548989452ec6774c8b6dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "731b82d6054548989452ec6774c8b6dd" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44573 } } }
I20260811 00:12:38.949185 11776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:38.949829 11777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 731b82d6054548989452ec6774c8b6dd. Latest consensus state: current_term: 1 leader_uuid: "731b82d6054548989452ec6774c8b6dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "731b82d6054548989452ec6774c8b6dd" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44573 } } }
I20260811 00:12:38.950269 11777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:38.952106 11784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:12:38.957360 11784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:12:38.966219 11784 catalog_manager.cc:1347] Generated new cluster ID: add229ed6336408abdbecbfa75b30ac5
I20260811 00:12:38.966324 11784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:12:38.978866 11784 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:12:38.980662 11784 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:12:38.993139 11784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 731b82d6054548989452ec6774c8b6dd: Generated new TSK 0
I20260811 00:12:38.994174 11784 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
I20260811 00:12:39.052441 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:44573
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:34023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:39.387140 11804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:39.387439 11804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:39.387532 11804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:39.394733 11804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:39.394838 11804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:39.394958 11804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:12:39.404335 11804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:34023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:44573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:39.405552 11804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:39.406804 11804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:39.420358 11817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:39.420392 11814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:39.420460 11813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:39.421160 11804 server_base.cc:1034] running on GCE node
I20260811 00:12:39.421850 11804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:39.422735 11804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:39.423981 11804 hybrid_clock.cc:648] HybridClock initialized: now 1786407159423910 us; error 64 us; skew 500 ppm
I20260811 00:12:39.424335 11804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:39 dist-test-slave-8hhm krb5kdc[11643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407159, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:39.427564 11804 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:12:39.428994 11804 webserver.cc:460] Webserver started at http://127.7.94.1:41897/ using document root <none> and password file <none>
I20260811 00:12:39.429761 11804 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:39.429863 11804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:39.430115 11804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:39.432906 11804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "11b073072cd443309c689c3ac208b581"
format_stamp: "Formatted at 2026-08-11 00:12:39 on dist-test-slave-8hhm"
server_key: "ff376fdf4317b98e46178590441bf3a3"
server_key_iv: "0877efef8f8066b2d894900a3eb28642"
server_key_version: "encryptionkey@0"
I20260811 00:12:39.433712 11804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "11b073072cd443309c689c3ac208b581"
format_stamp: "Formatted at 2026-08-11 00:12:39 on dist-test-slave-8hhm"
server_key: "ff376fdf4317b98e46178590441bf3a3"
server_key_iv: "0877efef8f8066b2d894900a3eb28642"
server_key_version: "encryptionkey@0"
I20260811 00:12:39.439070 11804 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:12:39.442656 11826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:39.443943 11804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:12:39.444123 11804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "11b073072cd443309c689c3ac208b581"
format_stamp: "Formatted at 2026-08-11 00:12:39 on dist-test-slave-8hhm"
server_key: "ff376fdf4317b98e46178590441bf3a3"
server_key_iv: "0877efef8f8066b2d894900a3eb28642"
server_key_version: "encryptionkey@0"
I20260811 00:12:39.444334 11804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:39.460834 11804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:39.464939 11804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:39.465320 11804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:39.466341 11804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:39.467787 11804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:39.467861 11804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:39.468031 11804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:39.468115 11804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:39.501372 11804 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:36361
I20260811 00:12:39.501410 12008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:36361 every 8 connection(s)
I20260811 00:12:39.502800 11804 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:12:39.503152 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 11804
I20260811 00:12:39.503387 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:12:39.504230 7544 external_mini_cluster.cc:1428] Setting key d51d45f5693d93a46c3dafba6e31d989
Aug 11 00:12:39 dist-test-slave-8hhm krb5kdc[11643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407159, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:39.526654 12015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:39.505759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:33599 (local address 127.7.94.62:44573)
0811 00:12:39.506297 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:39.506305 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:39.507362 (+ 1057us) server_negotiation.cc:407] Connection header received
0811 00:12:39.509200 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:39.509205 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:39.509331 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:39.509461 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:39.511362 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:39.512594 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:39.514924 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:39.515636 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:39.515790 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:39.519608 (+ 3818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:39.519617 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:39.519624 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:39.519668 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:39.523296 (+ 3628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:39.523991 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:39.523999 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:39.524006 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:39.524167 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:39.524573 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:39.524579 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:39.524582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:39.524969 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:39.525130 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:39.525870 (+ 740us) server_negotiation.cc:299] Negotiation successful
0811 00:12:39.526132 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":265,"threads_started":1}
I20260811 00:12:39.527158 12016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:39.506414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:44573 (local address 127.7.94.1:33599)
0811 00:12:39.507213 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:39.507251 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:39.508929 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:39.509703 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:39.509710 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:39.510450 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:39.511160 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:39.511184 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:39.512772 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:39.512780 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:39.514731 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:39.514750 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:39.515878 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:39.515883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:39.516072 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:39.516791 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:39.516819 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:39.519385 (+ 2566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:39.523436 (+ 4051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:39.523444 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:39.523457 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:39.523867 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:39.524282 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:39.524285 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:39.524287 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:39.524456 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:39.525104 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:39.525110 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:39.525472 (+ 362us) client_negotiation.cc:769] Sending connection context
0811 00:12:39.525716 (+ 244us) client_negotiation.cc:240] Negotiation successful
0811 00:12:39.526079 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":348,"threads_started":1}
I20260811 00:12:39.528764 12010 heartbeater.cc:347] Connected to a master server at 127.7.94.62:44573
I20260811 00:12:39.529099 12010 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.94.2@KRBTEST.COM; defaulting to no policy
I20260811 00:12:39.529948 12010 heartbeater.cc:511] Master 127.7.94.62:44573 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.2@KRBTEST.COM" created.
I20260811 00:12:39.532015 11710 ts_manager.cc:194] Registered new tserver with Master: 11b073072cd443309c689c3ac208b581 (127.7.94.1:36361)
I20260811 00:12:39.533588 11710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:33599
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
I20260811 00:12:39.557315 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:44573
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:34023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:39.891466 12022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:39.891805 12022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:39.892001 12022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:39.899127 12022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:39.899237 12022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:39.899459 12022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:12:39.908803 12022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:34023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:44573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:39.910095 12022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:39.911432 12022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:39.925071 12033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:39.925155 12035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:39.925007 12038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:39.926186 12022 server_base.cc:1034] running on GCE node
I20260811 00:12:39.926776 12022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:39.927772 12022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:39.929051 12022 hybrid_clock.cc:648] HybridClock initialized: now 1786407159928998 us; error 40 us; skew 500 ppm
I20260811 00:12:39.929438 12022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:39 dist-test-slave-8hhm krb5kdc[11643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407159, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:39.932829 12022 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:12:39.934458 12022 webserver.cc:460] Webserver started at http://127.7.94.2:36041/ using document root <none> and password file <none>
I20260811 00:12:39.935221 12022 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:39.935379 12022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:39.935741 12022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:39.938849 12022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "17c911b511854a0487126c860a0bac5b"
format_stamp: "Formatted at 2026-08-11 00:12:39 on dist-test-slave-8hhm"
server_key: "27afb3e9e03d311cacf2b12ac9200c39"
server_key_iv: "211805994b41ba52ae6786170bcf5e51"
server_key_version: "encryptionkey@0"
I20260811 00:12:39.939697 12022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "17c911b511854a0487126c860a0bac5b"
format_stamp: "Formatted at 2026-08-11 00:12:39 on dist-test-slave-8hhm"
server_key: "27afb3e9e03d311cacf2b12ac9200c39"
server_key_iv: "211805994b41ba52ae6786170bcf5e51"
server_key_version: "encryptionkey@0"
I20260811 00:12:39.944880 12022 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 00:12:39.948830 12051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:39.950183 12022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:12:39.950353 12022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "17c911b511854a0487126c860a0bac5b"
format_stamp: "Formatted at 2026-08-11 00:12:39 on dist-test-slave-8hhm"
server_key: "27afb3e9e03d311cacf2b12ac9200c39"
server_key_iv: "211805994b41ba52ae6786170bcf5e51"
server_key_version: "encryptionkey@0"
I20260811 00:12:39.950590 12022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:39.970023 12022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:39.974211 12022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:39.974506 12022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:39.975414 12022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:39.976693 12022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:39.976797 12022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:39.976972 12022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:39.977041 12022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:40.010475 12022 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:35495
I20260811 00:12:40.010563 12232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:35495 every 8 connection(s)
I20260811 00:12:40.011806 12022 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:12:40.018576 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 12022
I20260811 00:12:40.018770 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:12:40.019203 7544 external_mini_cluster.cc:1428] Setting key 0d8599c3ca171b3686d89b00e30a2613
Aug 11 00:12:40 dist-test-slave-8hhm krb5kdc[11643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407159, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:40.031821 12015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:40.014390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:36831 (local address 127.7.94.62:44573)
0811 00:12:40.014719 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:40.014738 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:40.015904 (+ 1166us) server_negotiation.cc:407] Connection header received
0811 00:12:40.017769 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:40.017775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:40.017866 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:40.017978 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:40.019601 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:40.020300 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:40.021841 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:40.022263 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:40.022415 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:40.025877 (+ 3462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:40.025882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:40.025885 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:40.025916 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:40.028664 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:40.029496 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:40.029501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:40.029504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:40.029625 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:40.030099 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:40.030134 (+ 35us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:40.030136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:40.030397 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:40.030574 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:40.031239 (+ 665us) server_negotiation.cc:299] Negotiation successful
0811 00:12:40.031468 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:12:40.032785 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:40.015028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:44573 (local address 127.7.94.2:36831)
0811 00:12:40.015740 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:40.015783 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:40.017506 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:40.018221 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:40.018229 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:40.018797 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:40.019438 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:40.019457 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:40.020457 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:40.020466 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:40.021705 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:40.021717 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:40.022527 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:40.022535 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:40.022749 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:40.023412 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:40.023441 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:40.025663 (+ 2222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:40.028818 (+ 3155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:40.028826 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:40.028841 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:40.029341 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:40.029758 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:40.029763 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:40.029766 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:40.029977 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:40.030548 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:40.030556 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:40.030884 (+ 328us) client_negotiation.cc:769] Sending connection context
0811 00:12:40.031136 (+ 252us) client_negotiation.cc:240] Negotiation successful
0811 00:12:40.031639 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":265,"threads_started":1}
I20260811 00:12:40.034217 12234 heartbeater.cc:347] Connected to a master server at 127.7.94.62:44573
I20260811 00:12:40.034511 12234 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:40.035192 12234 heartbeater.cc:511] Master 127.7.94.62:44573 requested a full tablet report, sending...
I20260811 00:12:40.036693 11710 ts_manager.cc:194] Registered new tserver with Master: 17c911b511854a0487126c860a0bac5b (127.7.94.2:35495)
I20260811 00:12:40.038000 11710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:36831
WARNING: no policy specified for kudu/127.7.94.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
I20260811 00:12:40.067857 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:44573
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:34023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:40.397570 12242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:40.397848 12242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:40.397943 12242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:40.404956 12242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:40.405072 12242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:40.405190 12242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:12:40.414773 12242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:34023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:44573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:40.416203 12242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:40.417536 12242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:40.430873 12248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:40.430962 12252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:40.430936 12247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:40.431732 12242 server_base.cc:1034] running on GCE node
I20260811 00:12:40.432301 12242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:40.433183 12242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:40.434456 12242 hybrid_clock.cc:648] HybridClock initialized: now 1786407160434418 us; error 29 us; skew 500 ppm
I20260811 00:12:40.434841 12242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:40 dist-test-slave-8hhm krb5kdc[11643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407160, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:40.438212 12242 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:12:40.439738 12242 webserver.cc:460] Webserver started at http://127.7.94.3:39521/ using document root <none> and password file <none>
I20260811 00:12:40.440443 12242 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:40.440580 12242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:40.440927 12242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:40.443933 12242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "a8f23cc791174825899d460b67ee0ca5"
format_stamp: "Formatted at 2026-08-11 00:12:40 on dist-test-slave-8hhm"
server_key: "151da341e97a7753cd4ca2ef84bb4a47"
server_key_iv: "b8eed28d00c5aff8da8b1c816194e1a7"
server_key_version: "encryptionkey@0"
I20260811 00:12:40.444736 12242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "a8f23cc791174825899d460b67ee0ca5"
format_stamp: "Formatted at 2026-08-11 00:12:40 on dist-test-slave-8hhm"
server_key: "151da341e97a7753cd4ca2ef84bb4a47"
server_key_iv: "b8eed28d00c5aff8da8b1c816194e1a7"
server_key_version: "encryptionkey@0"
I20260811 00:12:40.450335 12242 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260811 00:12:40.453982 12264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:40.455317 12242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:12:40.455524 12242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "a8f23cc791174825899d460b67ee0ca5"
format_stamp: "Formatted at 2026-08-11 00:12:40 on dist-test-slave-8hhm"
server_key: "151da341e97a7753cd4ca2ef84bb4a47"
server_key_iv: "b8eed28d00c5aff8da8b1c816194e1a7"
server_key_version: "encryptionkey@0"
I20260811 00:12:40.455719 12242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:40.476572 12242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:40.480456 12242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:40.480754 12242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:40.481758 12242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:40.483049 12242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:40.483135 12242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:40.483314 12242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:40.483367 12242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:40.516995 12242 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:35299
I20260811 00:12:40.517087 12447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:35299 every 8 connection(s)
I20260811 00:12:40.518471 12242 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:12:40.527699 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 12242
I20260811 00:12:40.527895 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:12:40.528321 7544 external_mini_cluster.cc:1428] Setting key 3f37896bc3505d79e76688c5ae91606d
Aug 11 00:12:40 dist-test-slave-8hhm krb5kdc[11643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407160, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:40.537072 12010 heartbeater.cc:503] Master 127.7.94.62:44573 was elected leader, sending a full tablet report...
I20260811 00:12:40.538036 12015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:40.521043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:49741 (local address 127.7.94.62:44573)
0811 00:12:40.521287 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:40.521292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:40.522475 (+ 1183us) server_negotiation.cc:407] Connection header received
0811 00:12:40.524284 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:40.524291 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:40.524381 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:40.524493 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:40.526048 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:40.526831 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:40.528524 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:40.529065 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:40.529210 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:40.532629 (+ 3419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:40.532637 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:40.532646 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:40.532678 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:40.535026 (+ 2348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:40.535739 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:40.535748 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:40.535750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:40.535875 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:40.536307 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:40.536311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:40.536313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:40.536557 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:40.536718 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:40.537464 (+ 746us) server_negotiation.cc:299] Negotiation successful
0811 00:12:40.537679 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 00:12:40.539019 12450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:40.521599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:44573 (local address 127.7.94.3:49741)
0811 00:12:40.522318 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:40.522356 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:40.524016 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:40.524736 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:40.524744 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:40.525323 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:40.525877 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:40.525895 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:40.526993 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:40.526998 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:40.528352 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:40.528387 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:40.529324 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:40.529335 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:40.529630 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:40.530349 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:40.530371 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:40.532431 (+ 2060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:40.535172 (+ 2741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:40.535181 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:40.535196 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:40.535608 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:40.535989 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:40.535994 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:40.535997 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:40.536180 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:40.536692 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:40.536699 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:40.537029 (+ 330us) client_negotiation.cc:769] Sending connection context
0811 00:12:40.537344 (+ 315us) client_negotiation.cc:240] Negotiation successful
0811 00:12:40.537852 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":283,"threads_started":1}
I20260811 00:12:40.540338 12448 heartbeater.cc:347] Connected to a master server at 127.7.94.62:44573
I20260811 00:12:40.540695 12448 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:40.541486 12448 heartbeater.cc:511] Master 127.7.94.62:44573 requested a full tablet report, sending...
I20260811 00:12:40.543057 11710 ts_manager.cc:194] Registered new tserver with Master: a8f23cc791174825899d460b67ee0ca5 (127.7.94.3:35299)
I20260811 00:12:40.543725 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:12:40.543954 11710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:49741
W20260811 00:12:40.581334 12015 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:12:40.578835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39094 (local address 127.7.94.62:44573)
0811 00:12:40.579160 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:40.579167 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:40.579476 (+ 309us) server_negotiation.cc:407] Connection header received
0811 00:12:40.579617 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:40.579620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:40.579688 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:40.579777 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:40.581010 (+ 1233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39094: BlockingRecv error: recv got EOF from 127.0.0.1:39094 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 00:12:40.584213 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 11804
W20260811 00:12:40.587426 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 11804: Not authorized: Client connection negotiation failed: client connection to 127.7.94.1:36361: server requires authentication, but client does not have Kerberos credentials available
I20260811 00:12:40.587631 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 11804
I20260811 00:12:40.618754 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 12022
W20260811 00:12:40.621546 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 12022: Not authorized: Client connection negotiation failed: client connection to 127.7.94.2:35495: server requires authentication, but client does not have Kerberos credentials available
I20260811 00:12:40.621729 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 12022
I20260811 00:12:40.643879 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 12242
W20260811 00:12:40.646720 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 12242: Not authorized: Client connection negotiation failed: client connection to 127.7.94.3:35299: server requires authentication, but client does not have Kerberos credentials available
I20260811 00:12:40.646845 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 12242
I20260811 00:12:40.672539 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 11659
I20260811 00:12:40.673897 11755 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:40.777652 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 11659
2026-08-11T00:12:40Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3839 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:12:40 dist-test-slave-8hhm krb5kdc[12474](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:12:40 dist-test-slave-8hhm krb5kdc[12474](info): set up 2 sockets
Aug 11 00:12:40 dist-test-slave-8hhm krb5kdc[12474](info): commencing operation
krb5kdc: starting...
W20260811 00:12:42.110466 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.006s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:12:42 dist-test-slave-8hhm krb5kdc[12474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:12:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:12:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:12:42.321460 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:42071
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:43509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:42071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:42.642768 12499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:42.643034 12499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:42.643185 12499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:42.650051 12499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:12:42.650151 12499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:42.650264 12499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:42.650358 12499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:12:42.650447 12499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:12:42.659457 12499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:43509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:42071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:42071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:42.660666 12499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:42.661929 12499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:42.673853 12505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:42.673805 12506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:42.673895 12510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:42.674664 12499 server_base.cc:1034] running on GCE node
I20260811 00:12:42.675518 12499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:42.676779 12499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:42.678004 12499 hybrid_clock.cc:648] HybridClock initialized: now 1786407162677970 us; error 47 us; skew 500 ppm
I20260811 00:12:42.678347 12499 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:42 dist-test-slave-8hhm krb5kdc[12474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407162, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:42.686407 12499 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:12:42.687906 12499 webserver.cc:460] Webserver started at http://127.7.94.62:43815/ using document root <none> and password file <none>
I20260811 00:12:42.688578 12499 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:42.688728 12499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:42.689038 12499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:42.692006 12499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "ab133fa04e254f21909ad0d94fd547a0"
format_stamp: "Formatted at 2026-08-11 00:12:42 on dist-test-slave-8hhm"
server_key: "950556eeedf69268ea556b8cad082266"
server_key_iv: "4cfc981119f698a84ddf787aedd0ec0f"
server_key_version: "encryptionkey@0"
I20260811 00:12:42.692797 12499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "ab133fa04e254f21909ad0d94fd547a0"
format_stamp: "Formatted at 2026-08-11 00:12:42 on dist-test-slave-8hhm"
server_key: "950556eeedf69268ea556b8cad082266"
server_key_iv: "4cfc981119f698a84ddf787aedd0ec0f"
server_key_version: "encryptionkey@0"
I20260811 00:12:42.697831 12499 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 00:12:42.701134 12521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:42.702455 12499 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:12:42.702630 12499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "ab133fa04e254f21909ad0d94fd547a0"
format_stamp: "Formatted at 2026-08-11 00:12:42 on dist-test-slave-8hhm"
server_key: "950556eeedf69268ea556b8cad082266"
server_key_iv: "4cfc981119f698a84ddf787aedd0ec0f"
server_key_version: "encryptionkey@0"
I20260811 00:12:42.702824 12499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:42.714717 12499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:42.718520 12499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:42.718807 12499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:42.737314 12499 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:42071
I20260811 00:12:42.737339 12594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:42071 every 8 connection(s)
I20260811 00:12:42.738595 12499 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:12:42.739346 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 12499
I20260811 00:12:42.739557 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:12:42.740317 7544 external_mini_cluster.cc:1428] Setting key bf2f7cc4c7dcb842c07f41a68722084c
I20260811 00:12:42.742872 12595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 00:12:42 dist-test-slave-8hhm krb5kdc[12474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:42.752925 12595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0: Bootstrap starting.
I20260811 00:12:42.756201 12595 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:42.757318 12595 log.cc:824] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:42.759886 12595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0: No bootstrap required, opened a new log
I20260811 00:12:42.760828 12599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:42.742043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52420 (local address 127.7.94.62:42071)
0811 00:12:42.742821 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:42.742832 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:42.742876 (+ 44us) server_negotiation.cc:407] Connection header received
0811 00:12:42.744487 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:42.744507 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:42.744844 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:42.745266 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:42.746735 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:42.748007 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:42.749193 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:42.749854 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:42.750100 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:42.753881 (+ 3781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:42.753888 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:42.753900 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:42.753941 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:42.756925 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:42.757684 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:42.757690 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:42.757695 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:42.757865 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:42.758293 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:42.758300 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:42.758302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:42.758764 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:42.759048 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:42.759494 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 00:12:42.759856 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":331,"threads_started":1}
I20260811 00:12:42.764766 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:12:42.768217 12595 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab133fa04e254f21909ad0d94fd547a0" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42071 } }
I20260811 00:12:42.768560 12595 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:42.768676 12595 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab133fa04e254f21909ad0d94fd547a0, State: Initialized, Role: FOLLOWER
I20260811 00:12:42.769331 12595 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab133fa04e254f21909ad0d94fd547a0" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42071 } }
I20260811 00:12:42.769574 12595 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:12:42.769718 12595 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:12:42.769901 12595 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:42.772084 12595 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab133fa04e254f21909ad0d94fd547a0" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42071 } }
I20260811 00:12:42.772557 12595 leader_election.cc:302] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab133fa04e254f21909ad0d94fd547a0; no voters:
I20260811 00:12:42.773226 12595 leader_election.cc:288] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:12:42.773313 12603 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:42.774536 12603 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0 [term 1 LEADER]: Becoming Leader. State: Replica: ab133fa04e254f21909ad0d94fd547a0, State: Running, Role: LEADER
I20260811 00:12:42.775127 12603 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab133fa04e254f21909ad0d94fd547a0" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42071 } }
I20260811 00:12:42.775554 12595 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:12:42.777845 12606 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab133fa04e254f21909ad0d94fd547a0. Latest consensus state: current_term: 1 leader_uuid: "ab133fa04e254f21909ad0d94fd547a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab133fa04e254f21909ad0d94fd547a0" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42071 } } }
I20260811 00:12:42.778173 12606 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0 [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:42.778689 12604 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab133fa04e254f21909ad0d94fd547a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab133fa04e254f21909ad0d94fd547a0" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42071 } } }
I20260811 00:12:42.778995 12604 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0 [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:42.781858 12619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:12:42.788897 12619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:12:42.803489 12619 catalog_manager.cc:1347] Generated new cluster ID: fe73c196d3cd4f93ad6baa9c958ec1ee
I20260811 00:12:42.803622 12619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:12:42.821326 12599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:42.809525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52430 (local address 127.7.94.62:42071)
0811 00:12:42.809697 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:42.809703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:42.813008 (+ 3305us) server_negotiation.cc:407] Connection header received
0811 00:12:42.813257 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:42.813263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:42.813364 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:42.813487 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:42.814645 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:42.815847 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:42.817661 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:42.818318 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:42.818462 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:42.819574 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:42.819587 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:42.819591 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:12:42.820466 (+ 875us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:42.820518 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:12:42.820558 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:42.820668 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:42.820818 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 00:12:42.820928 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 00:12:42.828217 12619 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:12:42.829560 12619 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:12:42.837927 12619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab133fa04e254f21909ad0d94fd547a0: Generated new TSK 0
I20260811 00:12:42.838680 12619 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:12:42.847124 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 12499
I20260811 00:12:42.848341 12587 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:42.941128 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 12499
2026-08-11T00:12:42Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2153 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:12:43 dist-test-slave-8hhm krb5kdc[12664](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:12:43 dist-test-slave-8hhm krb5kdc[12664](info): set up 2 sockets
Aug 11 00:12:43 dist-test-slave-8hhm krb5kdc[12664](info): commencing operation
krb5kdc: starting...
W20260811 00:12:44.277367 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:12:44 dist-test-slave-8hhm krb5kdc[12664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:12:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:12:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:12:44.496234 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:37273
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:44089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:37273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:44.821348 12684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:44.821635 12684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:44.821805 12684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:44.829061 12684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:12:44.829177 12684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:44.829300 12684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:44.829396 12684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:12:44.829459 12684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:12:44.839320 12684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:44089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:37273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:37273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:44.840725 12684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:44.842134 12684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:44.853968 12689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:44.854456 12695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:44.854651 12690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:44.854966 12684 server_base.cc:1034] running on GCE node
I20260811 00:12:44.855790 12684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:44.857210 12684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:44.858560 12684 hybrid_clock.cc:648] HybridClock initialized: now 1786407164858515 us; error 48 us; skew 500 ppm
I20260811 00:12:44.858922 12684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:44 dist-test-slave-8hhm krb5kdc[12664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407164, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:44.861992 12684 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:12:44.863394 12684 webserver.cc:460] Webserver started at http://127.7.94.62:42991/ using document root <none> and password file <none>
I20260811 00:12:44.864008 12684 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:44.864109 12684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:44.864383 12684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:44.867471 12684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "5d12f546084942b7ba9ca1ae2cbf9ffe"
format_stamp: "Formatted at 2026-08-11 00:12:44 on dist-test-slave-8hhm"
server_key: "544252b97281caa3f6564364f11b0182"
server_key_iv: "a33853bd1039d58c327d60f0dfbe3170"
server_key_version: "encryptionkey@0"
I20260811 00:12:44.868194 12684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "5d12f546084942b7ba9ca1ae2cbf9ffe"
format_stamp: "Formatted at 2026-08-11 00:12:44 on dist-test-slave-8hhm"
server_key: "544252b97281caa3f6564364f11b0182"
server_key_iv: "a33853bd1039d58c327d60f0dfbe3170"
server_key_version: "encryptionkey@0"
I20260811 00:12:44.873431 12684 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 00:12:44.876878 12706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:44.878206 12684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:12:44.878335 12684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "5d12f546084942b7ba9ca1ae2cbf9ffe"
format_stamp: "Formatted at 2026-08-11 00:12:44 on dist-test-slave-8hhm"
server_key: "544252b97281caa3f6564364f11b0182"
server_key_iv: "a33853bd1039d58c327d60f0dfbe3170"
server_key_version: "encryptionkey@0"
I20260811 00:12:44.878520 12684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:44.897396 12684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:44.901966 12684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:44.902318 12684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:44.921187 12684 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:37273
I20260811 00:12:44.921232 12788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:37273 every 8 connection(s)
I20260811 00:12:44.922961 12684 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:12:44.926807 12791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:44.927958 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 12684
I20260811 00:12:44.928194 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:12:44.929070 7544 external_mini_cluster.cc:1428] Setting key 7e68789358abe089dc7c694edb312ba8
I20260811 00:12:44.941566 12791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe: Bootstrap starting.
Aug 11 00:12:44 dist-test-slave-8hhm krb5kdc[12664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:44.945555 12791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:44.946846 12791 log.cc:824] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:44.949828 12791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe: No bootstrap required, opened a new log
I20260811 00:12:44.952478 12795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:44.931122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42990 (local address 127.7.94.62:37273)
0811 00:12:44.931965 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:44.931977 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:44.932015 (+ 38us) server_negotiation.cc:407] Connection header received
0811 00:12:44.933663 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:44.933686 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:44.934061 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:44.934385 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:44.936235 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:44.937630 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:44.939507 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:44.940154 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:44.940285 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:44.945008 (+ 4723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:44.945016 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:44.945030 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:44.945070 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:44.948403 (+ 3333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:44.949062 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:44.949077 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:44.949083 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:44.949326 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:44.949723 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:44.949728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:44.949731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:44.950298 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:44.950640 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:44.951052 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 00:12:44.951488 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":401,"threads_started":1}
I20260811 00:12:44.958061 12791 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d12f546084942b7ba9ca1ae2cbf9ffe" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 37273 } }
I20260811 00:12:44.958534 12791 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:44.958690 12791 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d12f546084942b7ba9ca1ae2cbf9ffe, State: Initialized, Role: FOLLOWER
I20260811 00:12:44.959316 12791 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d12f546084942b7ba9ca1ae2cbf9ffe" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 37273 } }
I20260811 00:12:44.959555 12791 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:12:44.959702 12791 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:12:44.959841 12791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:44.962319 12791 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d12f546084942b7ba9ca1ae2cbf9ffe" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 37273 } }
I20260811 00:12:44.962805 12791 leader_election.cc:302] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d12f546084942b7ba9ca1ae2cbf9ffe; no voters:
I20260811 00:12:44.963384 12791 leader_election.cc:288] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:12:44.963510 12798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:44.964279 12798 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe [term 1 LEADER]: Becoming Leader. State: Replica: 5d12f546084942b7ba9ca1ae2cbf9ffe, State: Running, Role: LEADER
I20260811 00:12:44.964849 12798 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d12f546084942b7ba9ca1ae2cbf9ffe" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 37273 } }
I20260811 00:12:44.965544 12791 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:12:44.967687 12799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d12f546084942b7ba9ca1ae2cbf9ffe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d12f546084942b7ba9ca1ae2cbf9ffe" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 37273 } } }
I20260811 00:12:44.967603 12800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d12f546084942b7ba9ca1ae2cbf9ffe. Latest consensus state: current_term: 1 leader_uuid: "5d12f546084942b7ba9ca1ae2cbf9ffe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d12f546084942b7ba9ca1ae2cbf9ffe" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 37273 } } }
I20260811 00:12:44.968088 12799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:44.968151 12800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:44.971860 12804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:12:44.976531 12804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:12:44.987383 12804 catalog_manager.cc:1347] Generated new cluster ID: 37eceb9f25014a6bbd280a12bf1a3f13
I20260811 00:12:44.987521 12804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:12:45.016022 12804 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:12:45.017997 12804 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:12:45.027696 12804 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d12f546084942b7ba9ca1ae2cbf9ffe: Generated new TSK 0
I20260811 00:12:45.028704 12804 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
I20260811 00:12:45.088734 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:37273
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:44089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:45.429194 12826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:45.429486 12826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:45.429579 12826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:45.436553 12826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:45.436654 12826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:45.436764 12826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:12:45.446106 12826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:44089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:37273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:45.447307 12826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:45.448565 12826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:45.462122 12837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:45.462324 12840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:45.462615 12843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:45.463379 12826 server_base.cc:1034] running on GCE node
I20260811 00:12:45.464004 12826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:45.464866 12826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:45.466068 12826 hybrid_clock.cc:648] HybridClock initialized: now 1786407165465997 us; error 59 us; skew 500 ppm
I20260811 00:12:45.466401 12826 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:45 dist-test-slave-8hhm krb5kdc[12664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407165, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:45.469650 12826 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:12:45.471031 12826 webserver.cc:460] Webserver started at http://127.7.94.1:45499/ using document root <none> and password file <none>
I20260811 00:12:45.471762 12826 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:45.471911 12826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:45.472239 12826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:45.475157 12826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "c99f44c3de5c441cb2a6f64715568fc7"
format_stamp: "Formatted at 2026-08-11 00:12:45 on dist-test-slave-8hhm"
server_key: "eb50068e74bc696a05a8147152cd9b4b"
server_key_iv: "6a022643590cad4cc00ba3ccb7d6a059"
server_key_version: "encryptionkey@0"
I20260811 00:12:45.475991 12826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "c99f44c3de5c441cb2a6f64715568fc7"
format_stamp: "Formatted at 2026-08-11 00:12:45 on dist-test-slave-8hhm"
server_key: "eb50068e74bc696a05a8147152cd9b4b"
server_key_iv: "6a022643590cad4cc00ba3ccb7d6a059"
server_key_version: "encryptionkey@0"
I20260811 00:12:45.481092 12826 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 00:12:45.484603 12856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:45.486027 12826 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:12:45.486229 12826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "c99f44c3de5c441cb2a6f64715568fc7"
format_stamp: "Formatted at 2026-08-11 00:12:45 on dist-test-slave-8hhm"
server_key: "eb50068e74bc696a05a8147152cd9b4b"
server_key_iv: "6a022643590cad4cc00ba3ccb7d6a059"
server_key_version: "encryptionkey@0"
I20260811 00:12:45.486433 12826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:45.505587 12826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:45.509527 12826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:45.509826 12826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:45.510792 12826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:45.512156 12826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:45.512260 12826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:45.512413 12826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:45.512488 12826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:45.546432 12826 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:43523
I20260811 00:12:45.546489 13046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:43523 every 8 connection(s)
I20260811 00:12:45.547763 12826 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:12:45.550964 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 12826
I20260811 00:12:45.551195 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:12:45.552116 7544 external_mini_cluster.cc:1428] Setting key c17a2ca45e9643402f823e5b78e7b161
Aug 11 00:12:45 dist-test-slave-8hhm krb5kdc[12664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407165, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:45.568593 13049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:45.550413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:42851 (local address 127.7.94.62:37273)
0811 00:12:45.550894 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:45.550899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:45.552004 (+ 1105us) server_negotiation.cc:407] Connection header received
0811 00:12:45.554197 (+ 2193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:45.554202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:45.554302 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:45.554435 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:45.555974 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:45.556727 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:45.558369 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:45.558795 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:45.558932 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:45.562559 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:45.562566 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:45.562571 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:45.562611 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:45.565169 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:45.566187 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:45.566190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:45.566193 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:45.566341 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:45.566839 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:45.566842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:45.566844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:45.567106 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:45.567218 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:45.568165 (+ 947us) server_negotiation.cc:299] Negotiation successful
0811 00:12:45.568334 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":260,"threads_started":1}
I20260811 00:12:45.570194 13055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:45.551010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:37273 (local address 127.7.94.1:42851)
0811 00:12:45.551838 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:45.551876 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:45.553927 (+ 2051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:45.554732 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:45.554740 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:45.555256 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:45.555807 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:45.555824 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:45.556897 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:45.556901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:45.558201 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:45.558214 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:45.559042 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:45.559050 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:45.559267 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:45.559956 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:45.559982 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:45.562348 (+ 2366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:45.565356 (+ 3008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:45.565367 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:45.565385 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:45.566052 (+ 667us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:45.566461 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:45.566468 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:45.566474 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:45.566723 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:45.567247 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:45.567259 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:45.567737 (+ 478us) client_negotiation.cc:769] Sending connection context
0811 00:12:45.568052 (+ 315us) client_negotiation.cc:240] Negotiation successful
0811 00:12:45.568644 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":343,"threads_started":1}
I20260811 00:12:45.571904 13047 heartbeater.cc:347] Connected to a master server at 127.7.94.62:37273
I20260811 00:12:45.572261 13047 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:45.573030 13047 heartbeater.cc:511] Master 127.7.94.62:37273 requested a full tablet report, sending...
I20260811 00:12:45.575516 12731 ts_manager.cc:194] Registered new tserver with Master: c99f44c3de5c441cb2a6f64715568fc7 (127.7.94.1:43523)
WARNING: no policy specified for kudu/127.7.94.2@KRBTEST.COM; defaulting to no policy
I20260811 00:12:45.577514 12731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:42851
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
I20260811 00:12:45.606154 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:37273
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:44089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:45.948868 13058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:45.949155 13058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:45.949380 13058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:45.956535 13058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:45.956668 13058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:45.956848 13058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:12:45.966869 13058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:44089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:37273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:45.968130 13058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:45.969515 13058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:45.982738 13067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:45.982869 13069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:45.983219 13065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:45.983577 13058 server_base.cc:1034] running on GCE node
I20260811 00:12:45.984210 13058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:45.985149 13058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:45.986393 13058 hybrid_clock.cc:648] HybridClock initialized: now 1786407165986332 us; error 49 us; skew 500 ppm
I20260811 00:12:45.986706 13058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:45 dist-test-slave-8hhm krb5kdc[12664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407165, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:45.990087 13058 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:12:45.991575 13058 webserver.cc:460] Webserver started at http://127.7.94.2:35517/ using document root <none> and password file <none>
I20260811 00:12:45.992276 13058 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:45.992425 13058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:45.992734 13058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:45.995651 13058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "a10594cb17364e9f814d446bd86e7e77"
format_stamp: "Formatted at 2026-08-11 00:12:45 on dist-test-slave-8hhm"
server_key: "622be4f4f88cd055e3e5940a6b24c497"
server_key_iv: "a99776a492d72e4bdfdc7311435f6f32"
server_key_version: "encryptionkey@0"
I20260811 00:12:45.996443 13058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "a10594cb17364e9f814d446bd86e7e77"
format_stamp: "Formatted at 2026-08-11 00:12:45 on dist-test-slave-8hhm"
server_key: "622be4f4f88cd055e3e5940a6b24c497"
server_key_iv: "a99776a492d72e4bdfdc7311435f6f32"
server_key_version: "encryptionkey@0"
I20260811 00:12:46.001571 13058 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 00:12:46.004945 13081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:46.006295 13058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:12:46.006469 13058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "a10594cb17364e9f814d446bd86e7e77"
format_stamp: "Formatted at 2026-08-11 00:12:45 on dist-test-slave-8hhm"
server_key: "622be4f4f88cd055e3e5940a6b24c497"
server_key_iv: "a99776a492d72e4bdfdc7311435f6f32"
server_key_version: "encryptionkey@0"
I20260811 00:12:46.006671 13058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:46.027994 13058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:46.031872 13058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:46.032172 13058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:46.033100 13058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:46.034404 13058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:46.034515 13058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:46.034672 13058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:46.034746 13058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:46.067322 13058 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:38685
I20260811 00:12:46.067370 13271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:38685 every 8 connection(s)
I20260811 00:12:46.068602 13058 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:12:46.069799 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 13058
I20260811 00:12:46.069990 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:12:46.070490 7544 external_mini_cluster.cc:1428] Setting key 4801ceded2a6fa7fc9cfbe20410eeebd
Aug 11 00:12:46 dist-test-slave-8hhm krb5kdc[12664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407165, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:46.094080 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:46.071808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:44579 (local address 127.7.94.62:37273)
0811 00:12:46.072274 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:46.072280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:46.073489 (+ 1209us) server_negotiation.cc:407] Connection header received
0811 00:12:46.076478 (+ 2989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:46.076482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:46.076550 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:46.076698 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:46.078687 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:46.079476 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:46.081898 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:46.082644 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:46.082791 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:46.086751 (+ 3960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:46.086760 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:46.086768 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:46.086815 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:46.090611 (+ 3796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:46.091428 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:46.091436 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:46.091443 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:46.091607 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:46.092049 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:46.092056 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:46.092061 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:46.092455 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:46.092625 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:46.093646 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0811 00:12:46.093816 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":223,"threads_started":1}
I20260811 00:12:46.095772 13280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:46.072349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:37273 (local address 127.7.94.2:44579)
0811 00:12:46.073312 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:46.073375 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:46.076166 (+ 2791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:46.076941 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:46.076949 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:46.077589 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:46.078476 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:46.078505 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:46.079653 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:46.079662 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:46.081741 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:46.081754 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:46.082850 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:46.082855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:46.083161 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:46.083917 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:46.083940 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:46.086492 (+ 2552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:46.090748 (+ 4256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:46.090756 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:46.090770 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:46.091302 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:46.091736 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:46.091739 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:46.091742 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:46.091923 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:46.092615 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:46.092625 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:46.093159 (+ 534us) client_negotiation.cc:769] Sending connection context
0811 00:12:46.093566 (+ 407us) client_negotiation.cc:240] Negotiation successful
0811 00:12:46.094231 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":390,"threads_started":1}
WARNING: no policy specified for kudu/127.7.94.3@KRBTEST.COM; defaulting to no policy
I20260811 00:12:46.097296 13273 heartbeater.cc:347] Connected to a master server at 127.7.94.62:37273
I20260811 00:12:46.097630 13273 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:46.098361 13273 heartbeater.cc:511] Master 127.7.94.62:37273 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.3@KRBTEST.COM" created.
I20260811 00:12:46.100021 12731 ts_manager.cc:194] Registered new tserver with Master: a10594cb17364e9f814d446bd86e7e77 (127.7.94.2:38685)
I20260811 00:12:46.100843 12731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:44579
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
I20260811 00:12:46.126215 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:37273
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:44089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:46.457738 13283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:46.458055 13283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:46.458223 13283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:46.465335 13283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:46.465474 13283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:46.465651 13283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:12:46.475338 13283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:44089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:37273
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:46.476879 13283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:46.478281 13283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:46.492002 13297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:46.492024 13295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:46.492250 13293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:46.492754 13283 server_base.cc:1034] running on GCE node
I20260811 00:12:46.493460 13283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:46.494309 13283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:46.495509 13283 hybrid_clock.cc:648] HybridClock initialized: now 1786407166495469 us; error 31 us; skew 500 ppm
I20260811 00:12:46.495821 13283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:46 dist-test-slave-8hhm krb5kdc[12664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407166, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:46.499099 13283 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:12:46.500532 13283 webserver.cc:460] Webserver started at http://127.7.94.3:35423/ using document root <none> and password file <none>
I20260811 00:12:46.501281 13283 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:46.501428 13283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:46.501767 13283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:46.504659 13283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "0e69946431214bb2bc78328f8c02ec5e"
format_stamp: "Formatted at 2026-08-11 00:12:46 on dist-test-slave-8hhm"
server_key: "3db41f24d7463a91f98d00d9406d2406"
server_key_iv: "307ed3b232ccd1a32b1423cf0fc299d5"
server_key_version: "encryptionkey@0"
I20260811 00:12:46.505503 13283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "0e69946431214bb2bc78328f8c02ec5e"
format_stamp: "Formatted at 2026-08-11 00:12:46 on dist-test-slave-8hhm"
server_key: "3db41f24d7463a91f98d00d9406d2406"
server_key_iv: "307ed3b232ccd1a32b1423cf0fc299d5"
server_key_version: "encryptionkey@0"
I20260811 00:12:46.510471 13283 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 00:12:46.513926 13306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:46.515232 13283 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:12:46.515349 13283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "0e69946431214bb2bc78328f8c02ec5e"
format_stamp: "Formatted at 2026-08-11 00:12:46 on dist-test-slave-8hhm"
server_key: "3db41f24d7463a91f98d00d9406d2406"
server_key_iv: "307ed3b232ccd1a32b1423cf0fc299d5"
server_key_version: "encryptionkey@0"
I20260811 00:12:46.515494 13283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:46.539069 13283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:46.543251 13283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:46.543488 13283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:46.544338 13283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:46.545655 13283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:46.545763 13283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:46.545953 13283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:46.546027 13283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:46.580019 13283 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:40657
I20260811 00:12:46.580106 13491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:40657 every 8 connection(s)
I20260811 00:12:46.580544 13047 heartbeater.cc:503] Master 127.7.94.62:37273 was elected leader, sending a full tablet report...
I20260811 00:12:46.581521 13283 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:12:46.588136 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 13283
I20260811 00:12:46.588335 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:12:46.588774 7544 external_mini_cluster.cc:1428] Setting key 179e350efd6c10bbd3a72af36a470e2c
Aug 11 00:12:46 dist-test-slave-8hhm krb5kdc[12664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407166, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:46.601348 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:46.584175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:40451 (local address 127.7.94.62:37273)
0811 00:12:46.584376 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:46.584382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:46.585600 (+ 1218us) server_negotiation.cc:407] Connection header received
0811 00:12:46.587341 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:46.587345 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:46.587419 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:46.587505 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:46.589052 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:46.590080 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:46.592061 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:46.592525 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:46.592674 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:46.595935 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:46.595941 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:46.595946 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:46.595987 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:46.598478 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:46.599170 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:46.599177 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:46.599180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:46.599297 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:46.599721 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:46.599726 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:46.599729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:46.600016 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:46.600186 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:46.600805 (+ 619us) server_negotiation.cc:299] Negotiation successful
0811 00:12:46.601016 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 00:12:46.602186 13494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:46.584719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:37273 (local address 127.7.94.3:40451)
0811 00:12:46.585454 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:46.585497 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:46.587070 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:46.587701 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:46.587708 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:46.588249 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:46.588884 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:46.588908 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:46.590291 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:46.590295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:46.591908 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:46.591919 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:46.592765 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:46.592773 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:46.592988 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:46.593615 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:46.593636 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:46.595745 (+ 2109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:46.598620 (+ 2875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:46.598628 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:46.598642 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:46.599043 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:46.599405 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:46.599409 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:46.599413 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:46.599582 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:46.600174 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:46.600181 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:46.600457 (+ 276us) client_negotiation.cc:769] Sending connection context
0811 00:12:46.600708 (+ 251us) client_negotiation.cc:240] Negotiation successful
0811 00:12:46.601191 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":260,"threads_started":1}
I20260811 00:12:46.603760 13492 heartbeater.cc:347] Connected to a master server at 127.7.94.62:37273
I20260811 00:12:46.604126 13492 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:46.604802 13492 heartbeater.cc:511] Master 127.7.94.62:37273 requested a full tablet report, sending...
I20260811 00:12:46.606215 12731 ts_manager.cc:194] Registered new tserver with Master: 0e69946431214bb2bc78328f8c02ec5e (127.7.94.3:40657)
I20260811 00:12:46.606953 12731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:40451
I20260811 00:12:46.616209 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 00:12:46 dist-test-slave-8hhm krb5kdc[12664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407166, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 00:12:46 dist-test-slave-8hhm krb5kdc[12664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407166, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:46.668344 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:46.655261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42996 (local address 127.7.94.62:37273)
0811 00:12:46.655522 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:46.655528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:46.655758 (+ 230us) server_negotiation.cc:407] Connection header received
0811 00:12:46.655905 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:46.655909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:46.655972 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:46.656076 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:46.657451 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:46.658197 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:46.659496 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:46.659858 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:46.659948 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:46.663642 (+ 3694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:46.663647 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:46.663650 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:46.663678 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:46.666036 (+ 2358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:46.666589 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:46.666592 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:46.666595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:46.666719 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:46.667120 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:46.667123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:46.667125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:46.667416 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:46.667622 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:46.667821 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 00:12:46.667973 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
W20260811 00:12:46.668946 12731 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42996
I20260811 00:12:46.670771 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 12826
Aug 11 00:12:46 dist-test-slave-8hhm krb5kdc[12664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407166, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:12:46.685644 13510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:46.671612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33206 (local address 127.7.94.1:43523)
0811 00:12:46.672170 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:46.672178 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:46.672201 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:12:46.672255 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:46.672261 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:46.672457 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:46.672576 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:46.673732 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:46.674687 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:46.675873 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:46.676399 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:46.676493 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:46.680068 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:46.680079 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:46.680092 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:46.680124 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:46.682933 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:46.683563 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:46.683573 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:46.683576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:46.683727 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:46.684105 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:46.684113 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:46.684116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:46.684439 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:46.684758 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:46.684991 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 00:12:46.685289 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":270,"threads_started":1}
W20260811 00:12:46.686892 13042 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33206
W20260811 00:12:46.687963 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 12826: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:12:46.688164 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 12826
I20260811 00:12:46.720535 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 13058
Aug 11 00:12:46 dist-test-slave-8hhm krb5kdc[12664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407166, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.94.2@KRBTEST.COM
I20260811 00:12:46.735971 13515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:46.721457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43648 (local address 127.7.94.2:38685)
0811 00:12:46.722027 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:46.722034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:46.722060 (+ 26us) server_negotiation.cc:407] Connection header received
0811 00:12:46.722120 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:46.722127 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:46.722337 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:46.722454 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:46.723600 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:46.724625 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:46.725935 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:46.726472 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:46.726568 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:46.730641 (+ 4073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:46.730653 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:46.730695 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:46.730747 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:46.733518 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:46.734112 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:46.734119 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:46.734121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:46.734251 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:46.734564 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:46.734566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:46.734568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:46.734900 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:46.735126 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:46.735377 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 00:12:46.735609 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":312,"threads_started":1}
W20260811 00:12:46.737342 13266 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43648
W20260811 00:12:46.738428 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 13058: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:12:46.738566 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 13058
I20260811 00:12:46.764814 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 13283
Aug 11 00:12:46 dist-test-slave-8hhm krb5kdc[12664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407166, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.94.3@KRBTEST.COM
I20260811 00:12:46.780417 13516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:46.765887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39666 (local address 127.7.94.3:40657)
0811 00:12:46.766563 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:46.766570 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:46.766599 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:12:46.766662 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:46.766668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:46.766871 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:46.767021 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:46.768257 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:46.769362 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:46.770489 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:46.770998 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:46.771088 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:46.775189 (+ 4101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:46.775200 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:46.775214 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:46.775249 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:46.778105 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:46.778716 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:46.778723 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:46.778725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:46.778855 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:46.779166 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:46.779170 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:46.779171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:46.779498 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:46.779722 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:46.779924 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 00:12:46.780112 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":362,"threads_started":1}
W20260811 00:12:46.781726 13483 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:39666
W20260811 00:12:46.782891 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 13283: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:12:46.783067 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 13283
I20260811 00:12:46.810067 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 12684
I20260811 00:12:46.811329 12781 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:46.918783 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 12684
2026-08-11T00:12:46Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3984 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:12:47 dist-test-slave-8hhm krb5kdc[13523](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:12:47 dist-test-slave-8hhm krb5kdc[13523](info): set up 2 sockets
Aug 11 00:12:47 dist-test-slave-8hhm krb5kdc[13523](info): commencing operation
krb5kdc: starting...
W20260811 00:12:48.240720 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:12:48 dist-test-slave-8hhm krb5kdc[13523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:12:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:12:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:12:48.450608 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:35489
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:35489
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:48.775097 13552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:48.775362 13552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:48.775473 13552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:48.782663 13552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:12:48.782788 13552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:48.782887 13552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:48.782979 13552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:12:48.783058 13552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:12:48.792564 13552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:35489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:35489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:48.793957 13552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:48.795264 13552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:48.807051 13557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:48.807099 13562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:48.807152 13558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:48.807955 13552 server_base.cc:1034] running on GCE node
I20260811 00:12:48.808676 13552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:48.810096 13552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:48.811298 13552 hybrid_clock.cc:648] HybridClock initialized: now 1786407168811256 us; error 29 us; skew 500 ppm
I20260811 00:12:48.811627 13552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:48 dist-test-slave-8hhm krb5kdc[13523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407168, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:48.819286 13552 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:12:48.820801 13552 webserver.cc:460] Webserver started at http://127.7.94.62:41103/ using document root <none> and password file <none>
I20260811 00:12:48.821571 13552 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:48.821708 13552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:48.822052 13552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:48.824975 13552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "461c25c40563416b8000b09e372122e1"
format_stamp: "Formatted at 2026-08-11 00:12:48 on dist-test-slave-8hhm"
server_key: "f0ecd77d7698b8d2e186256a0a96a1b5"
server_key_iv: "ad0bd76f6564160b2b35b2e0f5835526"
server_key_version: "encryptionkey@0"
I20260811 00:12:48.825803 13552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "461c25c40563416b8000b09e372122e1"
format_stamp: "Formatted at 2026-08-11 00:12:48 on dist-test-slave-8hhm"
server_key: "f0ecd77d7698b8d2e186256a0a96a1b5"
server_key_iv: "ad0bd76f6564160b2b35b2e0f5835526"
server_key_version: "encryptionkey@0"
I20260811 00:12:48.830632 13552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:12:48.835107 13571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:48.836431 13552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 00:12:48.836577 13552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "461c25c40563416b8000b09e372122e1"
format_stamp: "Formatted at 2026-08-11 00:12:48 on dist-test-slave-8hhm"
server_key: "f0ecd77d7698b8d2e186256a0a96a1b5"
server_key_iv: "ad0bd76f6564160b2b35b2e0f5835526"
server_key_version: "encryptionkey@0"
I20260811 00:12:48.836735 13552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:48.853118 13552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:48.856897 13552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:48.857127 13552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:48.875133 13552 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:35489
I20260811 00:12:48.875186 13643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:35489 every 8 connection(s)
I20260811 00:12:48.876526 13552 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:12:48.880124 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 13552
I20260811 00:12:48.880370 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:12:48.880704 13644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:48.881278 7544 external_mini_cluster.cc:1428] Setting key dac6fd575cb292f8cbac0f4020bc8b9f
I20260811 00:12:48.898875 13644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1: Bootstrap starting.
Aug 11 00:12:48 dist-test-slave-8hhm krb5kdc[13523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:48.902931 13644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:48.904444 13644 log.cc:824] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:48.907534 13644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1: No bootstrap required, opened a new log
I20260811 00:12:48.911194 13651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:48.886541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53364 (local address 127.7.94.62:35489)
0811 00:12:48.887411 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:48.887428 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:48.887483 (+ 55us) server_negotiation.cc:407] Connection header received
0811 00:12:48.890147 (+ 2664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:48.890185 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:48.890875 (+ 690us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:48.891350 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:48.893343 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:48.894669 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:48.896672 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:48.897410 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:48.897548 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:48.902503 (+ 4955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:48.902514 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:48.902533 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:48.902578 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:48.906489 (+ 3911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:48.907165 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:48.907171 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:48.907177 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:48.907346 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:48.907753 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:48.907758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:48.907762 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:48.908505 (+ 743us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:48.908858 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:48.909161 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 00:12:48.909670 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":362,"threads_started":1}
I20260811 00:12:48.917732 13644 raft_consensus.cc:348] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "461c25c40563416b8000b09e372122e1" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35489 } }
I20260811 00:12:48.918081 13644 raft_consensus.cc:374] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:48.918161 13644 raft_consensus.cc:729] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 461c25c40563416b8000b09e372122e1, State: Initialized, Role: FOLLOWER
I20260811 00:12:48.918730 13644 consensus_queue.cc:260] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "461c25c40563416b8000b09e372122e1" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35489 } }
I20260811 00:12:48.918959 13644 raft_consensus.cc:388] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:12:48.919073 13644 raft_consensus.cc:482] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:12:48.919200 13644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:48.921821 13644 raft_consensus.cc:504] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "461c25c40563416b8000b09e372122e1" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35489 } }
I20260811 00:12:48.922281 13644 leader_election.cc:302] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 461c25c40563416b8000b09e372122e1; no voters:
I20260811 00:12:48.922873 13644 leader_election.cc:288] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:12:48.923017 13656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:48.923812 13656 raft_consensus.cc:686] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1 [term 1 LEADER]: Becoming Leader. State: Replica: 461c25c40563416b8000b09e372122e1, State: Running, Role: LEADER
I20260811 00:12:48.924352 13656 consensus_queue.cc:237] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "461c25c40563416b8000b09e372122e1" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35489 } }
I20260811 00:12:48.925205 13644 sys_catalog.cc:564] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:12:48.927634 13657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "461c25c40563416b8000b09e372122e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "461c25c40563416b8000b09e372122e1" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35489 } } }
I20260811 00:12:48.927989 13657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1 [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:48.928424 13658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 461c25c40563416b8000b09e372122e1. Latest consensus state: current_term: 1 leader_uuid: "461c25c40563416b8000b09e372122e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "461c25c40563416b8000b09e372122e1" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35489 } } }
I20260811 00:12:48.928704 13658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1 [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:48.928969 13665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:12:48.936215 13665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:12:48.947342 13665 catalog_manager.cc:1347] Generated new cluster ID: 2bb32888692f4ba49a6a2a3a82411bd2
I20260811 00:12:48.947467 13665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:12:48.966017 13665 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:12:48.967331 13665 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:12:48.973529 13665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 461c25c40563416b8000b09e372122e1: Generated new TSK 0
I20260811 00:12:48.974277 13665 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
I20260811 00:12:49.029084 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:35489
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:49.361519 13686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:49.361793 13686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:49.361886 13686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:49.368837 13686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:49.368939 13686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:49.369055 13686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:12:49.378728 13686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:35489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:49.379953 13686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:49.381258 13686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:49.394140 13700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:49.394191 13701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:49.394321 13704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:49.395061 13686 server_base.cc:1034] running on GCE node
I20260811 00:12:49.395682 13686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:49.396648 13686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:49.397867 13686 hybrid_clock.cc:648] HybridClock initialized: now 1786407169397805 us; error 51 us; skew 500 ppm
I20260811 00:12:49.398185 13686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:49 dist-test-slave-8hhm krb5kdc[13523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407169, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:49.401386 13686 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:12:49.402802 13686 webserver.cc:460] Webserver started at http://127.7.94.1:46175/ using document root <none> and password file <none>
I20260811 00:12:49.403484 13686 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:49.403643 13686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:49.403957 13686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:49.406930 13686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "712671de8b3b4d15b35aafb6c4cbed28"
format_stamp: "Formatted at 2026-08-11 00:12:49 on dist-test-slave-8hhm"
server_key: "c42a0f8214dcff50b82d72e254efbb58"
server_key_iv: "e8dfd0b144a686fb0be91ced12f55aa6"
server_key_version: "encryptionkey@0"
I20260811 00:12:49.407739 13686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "712671de8b3b4d15b35aafb6c4cbed28"
format_stamp: "Formatted at 2026-08-11 00:12:49 on dist-test-slave-8hhm"
server_key: "c42a0f8214dcff50b82d72e254efbb58"
server_key_iv: "e8dfd0b144a686fb0be91ced12f55aa6"
server_key_version: "encryptionkey@0"
I20260811 00:12:49.412912 13686 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 00:12:49.416292 13717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:49.417630 13686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:12:49.417805 13686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "712671de8b3b4d15b35aafb6c4cbed28"
format_stamp: "Formatted at 2026-08-11 00:12:49 on dist-test-slave-8hhm"
server_key: "c42a0f8214dcff50b82d72e254efbb58"
server_key_iv: "e8dfd0b144a686fb0be91ced12f55aa6"
server_key_version: "encryptionkey@0"
I20260811 00:12:49.418036 13686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:49.441177 13686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:49.445094 13686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:49.445416 13686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:49.446342 13686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:49.447945 13686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:49.448042 13686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:49.448169 13686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:49.448253 13686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:49.481977 13686 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:40559
I20260811 00:12:49.482036 13905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:40559 every 8 connection(s)
I20260811 00:12:49.483258 13686 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:12:49.490298 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 13686
I20260811 00:12:49.490531 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:12:49.491381 7544 external_mini_cluster.cc:1428] Setting key ee0025a83ef6d57a920758c87ec59172
Aug 11 00:12:49 dist-test-slave-8hhm krb5kdc[13523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407169, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:49.505172 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:49.485776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:41599 (local address 127.7.94.62:35489)
0811 00:12:49.486180 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:49.486184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:49.487230 (+ 1046us) server_negotiation.cc:407] Connection header received
0811 00:12:49.489084 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:49.489089 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:49.489177 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:49.489337 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:49.491288 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:49.492183 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:49.494251 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:49.494641 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:49.494741 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:49.499216 (+ 4475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:49.499221 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:49.499224 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:49.499255 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:49.501988 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:49.502682 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:49.502690 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:49.502696 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:49.502853 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:49.503255 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:49.503263 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:49.503268 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:49.503633 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:49.503790 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:49.504413 (+ 623us) server_negotiation.cc:299] Negotiation successful
0811 00:12:49.504685 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":206,"threads_started":1}
I20260811 00:12:49.505769 13911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:49.486360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35489 (local address 127.7.94.1:41599)
0811 00:12:49.487069 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:49.487105 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:49.488829 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:49.489566 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:49.489575 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:49.490276 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:49.491088 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:49.491117 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:49.492338 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:49.492343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:49.494089 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:49.494116 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:49.494872 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:49.494883 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:49.495213 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:49.495970 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:49.496005 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:49.498980 (+ 2975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:49.502143 (+ 3163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:49.502150 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:49.502164 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:49.502565 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:49.502967 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:49.502970 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:49.502972 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:49.503144 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:49.503760 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:49.503765 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:49.504045 (+ 280us) client_negotiation.cc:769] Sending connection context
0811 00:12:49.504267 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 00:12:49.504640 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":288,"threads_started":1}
I20260811 00:12:49.507386 13907 heartbeater.cc:347] Connected to a master server at 127.7.94.62:35489
I20260811 00:12:49.507705 13907 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:49.508402 13907 heartbeater.cc:511] Master 127.7.94.62:35489 requested a full tablet report, sending...
I20260811 00:12:49.510479 13592 ts_manager.cc:194] Registered new tserver with Master: 712671de8b3b4d15b35aafb6c4cbed28 (127.7.94.1:40559)
I20260811 00:12:49.512014 13592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:41599
WARNING: no policy specified for kudu/127.7.94.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
I20260811 00:12:49.542625 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:35489
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:49.872673 13915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:49.872999 13915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:49.873172 13915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:49.880082 13915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:49.880228 13915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:49.880393 13915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:12:49.889842 13915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:35489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:49.891029 13915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:49.892318 13915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:49.905469 13920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:49.905405 13921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:49.905568 13923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:49.906283 13915 server_base.cc:1034] running on GCE node
I20260811 00:12:49.906865 13915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:49.907752 13915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:49.908981 13915 hybrid_clock.cc:648] HybridClock initialized: now 1786407169908895 us; error 79 us; skew 500 ppm
I20260811 00:12:49.909351 13915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:49 dist-test-slave-8hhm krb5kdc[13523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407169, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:49.912515 13915 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:12:49.913946 13915 webserver.cc:460] Webserver started at http://127.7.94.2:38803/ using document root <none> and password file <none>
I20260811 00:12:49.914613 13915 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:49.914767 13915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:49.915081 13915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:49.918144 13915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "75fc26f0adde4cba85f47e545a8d5fb4"
format_stamp: "Formatted at 2026-08-11 00:12:49 on dist-test-slave-8hhm"
server_key: "8c32c851ae6cfb8a647bba09ae532f77"
server_key_iv: "fd5ee95f74c388582081c8604788a1b2"
server_key_version: "encryptionkey@0"
I20260811 00:12:49.918938 13915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "75fc26f0adde4cba85f47e545a8d5fb4"
format_stamp: "Formatted at 2026-08-11 00:12:49 on dist-test-slave-8hhm"
server_key: "8c32c851ae6cfb8a647bba09ae532f77"
server_key_iv: "fd5ee95f74c388582081c8604788a1b2"
server_key_version: "encryptionkey@0"
I20260811 00:12:49.924114 13915 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:12:49.927554 13934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:49.928797 13915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 00:12:49.928970 13915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "75fc26f0adde4cba85f47e545a8d5fb4"
format_stamp: "Formatted at 2026-08-11 00:12:49 on dist-test-slave-8hhm"
server_key: "8c32c851ae6cfb8a647bba09ae532f77"
server_key_iv: "fd5ee95f74c388582081c8604788a1b2"
server_key_version: "encryptionkey@0"
I20260811 00:12:49.929184 13915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:49.954704 13915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:49.959022 13915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:49.959326 13915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:49.960266 13915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:49.961623 13915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:49.961730 13915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:49.961881 13915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:49.961954 13915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:49.995282 13915 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:39955
I20260811 00:12:49.995323 14106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:39955 every 8 connection(s)
I20260811 00:12:49.996575 13915 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:12:50.003489 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 13915
I20260811 00:12:50.003715 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:12:50.004230 7544 external_mini_cluster.cc:1428] Setting key a618e27b8446d1a04e5190238479055d
Aug 11 00:12:50 dist-test-slave-8hhm krb5kdc[13523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407169, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:50.018111 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:49.999217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:42405 (local address 127.7.94.62:35489)
0811 00:12:49.999426 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:49.999432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:50.000641 (+ 1209us) server_negotiation.cc:407] Connection header received
0811 00:12:50.002474 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:50.002479 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:50.002553 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:50.002643 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:50.004602 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:50.005418 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:50.007559 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:50.007972 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:50.008065 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:50.012385 (+ 4320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:50.012390 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:50.012393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:50.012423 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:50.015085 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:50.015823 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:50.015832 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:50.015835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:50.015958 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:50.016378 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:50.016384 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:50.016387 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:50.016660 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:50.016813 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:50.017539 (+ 726us) server_negotiation.cc:299] Negotiation successful
0811 00:12:50.017769 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:12:50.019120 14117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:49.999783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35489 (local address 127.7.94.2:42405)
0811 00:12:50.000478 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:50.000516 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:50.002227 (+ 1711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:50.002848 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:50.002857 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:50.003603 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:50.004426 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:50.004453 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:50.005566 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:50.005570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:50.007410 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:50.007427 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:50.008212 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:50.008223 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:50.008570 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:50.009342 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:50.009376 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:50.012169 (+ 2793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:50.015238 (+ 3069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:50.015248 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:50.015263 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:50.015694 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:50.016066 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:50.016071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:50.016075 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:50.016265 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:50.016794 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:50.016802 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:50.017124 (+ 322us) client_negotiation.cc:769] Sending connection context
0811 00:12:50.017448 (+ 324us) client_negotiation.cc:240] Negotiation successful
0811 00:12:50.017975 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":281,"threads_started":1}
I20260811 00:12:50.020606 14113 heartbeater.cc:347] Connected to a master server at 127.7.94.62:35489
I20260811 00:12:50.020908 14113 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:50.021710 14113 heartbeater.cc:511] Master 127.7.94.62:35489 requested a full tablet report, sending...
I20260811 00:12:50.023209 13592 ts_manager.cc:194] Registered new tserver with Master: 75fc26f0adde4cba85f47e545a8d5fb4 (127.7.94.2:39955)
I20260811 00:12:50.023989 13592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:42405
WARNING: no policy specified for kudu/127.7.94.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
I20260811 00:12:50.054672 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:35489
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:50.384737 14121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:50.384996 14121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:50.385087 14121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:50.392021 14121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:50.392123 14121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:50.392230 14121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:12:50.401453 14121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:35489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:50.402623 14121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:50.403945 14121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:50.416652 14128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:50.416607 14129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:50.416590 14133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:50.417690 14121 server_base.cc:1034] running on GCE node
I20260811 00:12:50.418335 14121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:50.419261 14121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:50.420493 14121 hybrid_clock.cc:648] HybridClock initialized: now 1786407170420443 us; error 62 us; skew 500 ppm
I20260811 00:12:50.420840 14121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:50 dist-test-slave-8hhm krb5kdc[13523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407170, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:50.424223 14121 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:12:50.425757 14121 webserver.cc:460] Webserver started at http://127.7.94.3:39323/ using document root <none> and password file <none>
I20260811 00:12:50.426446 14121 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:50.426599 14121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:50.426916 14121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:50.429873 14121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "cbc37d2a97044a9c92083b519e1af46a"
format_stamp: "Formatted at 2026-08-11 00:12:50 on dist-test-slave-8hhm"
server_key: "c8f5135df8201498fb8e7504c627f9bf"
server_key_iv: "d740de153e8befc641c22e6adce482c7"
server_key_version: "encryptionkey@0"
I20260811 00:12:50.430657 14121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "cbc37d2a97044a9c92083b519e1af46a"
format_stamp: "Formatted at 2026-08-11 00:12:50 on dist-test-slave-8hhm"
server_key: "c8f5135df8201498fb8e7504c627f9bf"
server_key_iv: "d740de153e8befc641c22e6adce482c7"
server_key_version: "encryptionkey@0"
I20260811 00:12:50.435984 14121 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:12:50.439414 14147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:50.440876 14121 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 00:12:50.441048 14121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "cbc37d2a97044a9c92083b519e1af46a"
format_stamp: "Formatted at 2026-08-11 00:12:50 on dist-test-slave-8hhm"
server_key: "c8f5135df8201498fb8e7504c627f9bf"
server_key_iv: "d740de153e8befc641c22e6adce482c7"
server_key_version: "encryptionkey@0"
I20260811 00:12:50.441282 14121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:50.464131 14121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:50.468075 14121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:50.468372 14121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:50.469336 14121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:50.470650 14121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:50.470748 14121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:50.470892 14121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:50.470963 14121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:50.504719 14121 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:42085
I20260811 00:12:50.504810 14334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:42085 every 8 connection(s)
I20260811 00:12:50.506058 14121 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:12:50.514838 13907 heartbeater.cc:503] Master 127.7.94.62:35489 was elected leader, sending a full tablet report...
I20260811 00:12:50.515857 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 14121
I20260811 00:12:50.516063 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:12:50.516618 7544 external_mini_cluster.cc:1428] Setting key e2df3977d20a3eb2d1a45f2eec0dd395
Aug 11 00:12:50 dist-test-slave-8hhm krb5kdc[13523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407170, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:50.526240 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:50.508581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:40589 (local address 127.7.94.62:35489)
0811 00:12:50.508765 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:50.508769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:50.509932 (+ 1163us) server_negotiation.cc:407] Connection header received
0811 00:12:50.511779 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:50.511785 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:50.511881 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:50.511993 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:50.513507 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:50.514294 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:50.516425 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:50.516964 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:50.517104 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:50.520814 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:50.520819 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:50.520822 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:50.520853 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:50.523378 (+ 2525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:50.524019 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:50.524022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:50.524024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:50.524244 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:50.524620 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:50.524623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:50.524625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:50.524913 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:50.525023 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:50.525822 (+ 799us) server_negotiation.cc:299] Negotiation successful
0811 00:12:50.525988 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 00:12:50.527266 14337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:50.508979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35489 (local address 127.7.94.3:40589)
0811 00:12:50.509762 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:50.509806 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:50.511497 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:50.512226 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:50.512234 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:50.512730 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:50.513332 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:50.513352 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:50.514455 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:50.514462 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:50.516283 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:50.516296 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:50.517187 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:50.517197 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:50.517515 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:50.518265 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:50.518306 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:50.520630 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:50.523506 (+ 2876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:50.523512 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:50.523523 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:50.523917 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:50.524342 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:50.524345 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:50.524347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:50.524520 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:50.525023 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:50.525028 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:50.525429 (+ 401us) client_negotiation.cc:769] Sending connection context
0811 00:12:50.525729 (+ 300us) client_negotiation.cc:240] Negotiation successful
0811 00:12:50.526105 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":346,"threads_started":1}
I20260811 00:12:50.528649 14335 heartbeater.cc:347] Connected to a master server at 127.7.94.62:35489
I20260811 00:12:50.528939 14335 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:50.529764 14335 heartbeater.cc:511] Master 127.7.94.62:35489 requested a full tablet report, sending...
I20260811 00:12:50.531479 13590 ts_manager.cc:194] Registered new tserver with Master: cbc37d2a97044a9c92083b519e1af46a (127.7.94.3:42085)
I20260811 00:12:50.532291 13590 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:40589
I20260811 00:12:50.543462 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 00:12:50 dist-test-slave-8hhm krb5kdc[13523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 00:12:50 dist-test-slave-8hhm krb5kdc[13523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:12:50.596572 14343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:50.582124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52624 (local address 127.7.94.1:40559)
0811 00:12:50.582804 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:50.582812 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:50.582839 (+ 27us) server_negotiation.cc:407] Connection header received
0811 00:12:50.582908 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:50.582914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:50.583131 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:50.583272 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:50.584555 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:50.585631 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:50.586776 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:50.587318 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:50.587414 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:50.591260 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:50.591271 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:50.591286 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:50.591322 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:50.594247 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:50.594814 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:50.594820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:50.594822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:50.594943 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:50.595241 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:50.595244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:50.595246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:50.595540 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:50.595748 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:50.595938 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 00:12:50.596145 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":342,"threads_started":1}
Aug 11 00:12:50 dist-test-slave-8hhm krb5kdc[13523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:50.613711 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:50.601171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53366 (local address 127.7.94.62:35489)
0811 00:12:50.601412 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:50.601417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:50.601487 (+ 70us) server_negotiation.cc:407] Connection header received
0811 00:12:50.601599 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:50.601605 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:50.601666 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:50.601743 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:50.602886 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:50.603581 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:50.604809 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:50.605192 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:50.605339 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:50.608796 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:50.608800 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:50.608803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:50.608832 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:50.611439 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:50.612055 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:50.612060 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:50.612063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:50.612204 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:50.612525 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:50.612528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:50.612530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:50.612816 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:50.613003 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:50.613293 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 00:12:50.613466 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
W20260811 00:12:50.614236 13590 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53366
I20260811 00:12:50.615237 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 13686
I20260811 00:12:50.628011 14343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:50.616041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52640 (local address 127.7.94.1:40559)
0811 00:12:50.616266 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:50.616272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:50.616495 (+ 223us) server_negotiation.cc:407] Connection header received
0811 00:12:50.616670 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:50.616675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:50.616770 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:50.616871 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:50.618049 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:50.618721 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:50.619883 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:50.620300 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:50.620430 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:50.622713 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:50.622720 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:50.622725 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:50.622758 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:50.625332 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:50.626050 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:50.626056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:50.626060 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:50.626192 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:50.626575 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:50.626581 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:50.626585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:50.626917 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:50.627169 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:50.627462 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 00:12:50.627676 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 00:12:50.628582 13899 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:50.710682 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 13686
I20260811 00:12:50.735047 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 13915
Aug 11 00:12:50 dist-test-slave-8hhm krb5kdc[13523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.2@KRBTEST.COM
I20260811 00:12:50.750731 14353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:50.735864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50550 (local address 127.7.94.2:39955)
0811 00:12:50.736451 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:50.736458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:50.736485 (+ 27us) server_negotiation.cc:407] Connection header received
0811 00:12:50.736549 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:50.736554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:50.736798 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:50.736931 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:50.738231 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:50.739223 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:50.740460 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:50.741083 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:50.741183 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:50.745466 (+ 4283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:50.745479 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:50.745494 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:50.745525 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:50.748348 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:50.748991 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:50.748998 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:50.749000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:50.749129 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:50.749483 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:50.749486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:50.749488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:50.749817 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:50.750052 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:50.750253 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 00:12:50.750439 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":298,"threads_started":1}
I20260811 00:12:50.751973 14099 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:50.837705 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 13915
I20260811 00:12:50.865641 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 14121
Aug 11 00:12:50 dist-test-slave-8hhm krb5kdc[13523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.3@KRBTEST.COM
I20260811 00:12:50.886166 14356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:50.866712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50930 (local address 127.7.94.3:42085)
0811 00:12:50.867481 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:50.867489 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:50.867522 (+ 33us) server_negotiation.cc:407] Connection header received
0811 00:12:50.867594 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:50.867600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:50.867822 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:50.868016 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:50.869429 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:50.870487 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:50.871874 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:50.872487 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:50.872648 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:50.877847 (+ 5199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:50.877873 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:50.877888 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:50.877928 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:50.881395 (+ 3467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:50.882278 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:50.882324 (+ 46us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:50.882330 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:50.882546 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:50.882990 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:50.882998 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:50.883002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:50.883533 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:50.883870 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:50.885299 (+ 1429us) server_negotiation.cc:299] Negotiation successful
0811 00:12:50.885637 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":358,"threads_started":1}
I20260811 00:12:50.891997 14329 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:50.974298 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 14121
I20260811 00:12:51.002022 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 13552
I20260811 00:12:51.003247 13639 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:51.106846 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 13552
2026-08-11T00:12:51Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4187 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:12:51 dist-test-slave-8hhm krb5kdc[14366](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:12:51 dist-test-slave-8hhm krb5kdc[14366](info): set up 2 sockets
Aug 11 00:12:51 dist-test-slave-8hhm krb5kdc[14366](info): commencing operation
krb5kdc: starting...
W20260811 00:12:52.441835 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.004s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:12:52 dist-test-slave-8hhm krb5kdc[14366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:12:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:12:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:12:52.662294 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:36885
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:42133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:36885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:52.995190 14387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:52.995476 14387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:52.995565 14387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:53.002912 14387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:12:53.003031 14387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:53.003079 14387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:53.003137 14387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:12:53.003180 14387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:12:53.013012 14387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:42133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:36885
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:36885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:53.014369 14387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:53.015654 14387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:53.027827 14396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:53.027887 14397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:53.028357 14399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:53.028861 14387 server_base.cc:1034] running on GCE node
I20260811 00:12:53.029655 14387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:53.030931 14387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:53.032151 14387 hybrid_clock.cc:648] HybridClock initialized: now 1786407173032097 us; error 49 us; skew 500 ppm
I20260811 00:12:53.032578 14387 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:53 dist-test-slave-8hhm krb5kdc[14366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407173, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:53.035794 14387 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:12:53.037014 14387 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:53.037153 14387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:53.037501 14387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:53.040467 14387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "ef9402b911714fa680d81711a64c496b"
format_stamp: "Formatted at 2026-08-11 00:12:53 on dist-test-slave-8hhm"
server_key: "e70da1a4e837103ab4ea64b72e4cb3e0"
server_key_iv: "8aed18b24fc59180b89b53dd5e08b0f2"
server_key_version: "encryptionkey@0"
I20260811 00:12:53.041328 14387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "ef9402b911714fa680d81711a64c496b"
format_stamp: "Formatted at 2026-08-11 00:12:53 on dist-test-slave-8hhm"
server_key: "e70da1a4e837103ab4ea64b72e4cb3e0"
server_key_iv: "8aed18b24fc59180b89b53dd5e08b0f2"
server_key_version: "encryptionkey@0"
I20260811 00:12:53.046540 14387 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 00:12:53.049993 14411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:53.051326 14387 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:12:53.051529 14387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "ef9402b911714fa680d81711a64c496b"
format_stamp: "Formatted at 2026-08-11 00:12:53 on dist-test-slave-8hhm"
server_key: "e70da1a4e837103ab4ea64b72e4cb3e0"
server_key_iv: "8aed18b24fc59180b89b53dd5e08b0f2"
server_key_version: "encryptionkey@0"
I20260811 00:12:53.051806 14387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:53.074898 14387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:53.079200 14387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:53.079536 14387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:53.098300 14387 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:36885
I20260811 00:12:53.098320 14497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:36885 every 8 connection(s)
I20260811 00:12:53.099224 14387 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:12:53.102738 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 14387
I20260811 00:12:53.102838 14499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:53.103009 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:12:53.103653 7544 external_mini_cluster.cc:1428] Setting key cd278b8ec21d3a109ec04e9d046699ca
I20260811 00:12:53.117523 14499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b: Bootstrap starting.
Aug 11 00:12:53 dist-test-slave-8hhm krb5kdc[14366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:53.120813 14499 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:53.122224 14499 log.cc:824] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:53.125697 14499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b: No bootstrap required, opened a new log
I20260811 00:12:53.129317 14506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:53.105685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60814 (local address 127.7.94.62:36885)
0811 00:12:53.106577 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:53.106592 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:53.106649 (+ 57us) server_negotiation.cc:407] Connection header received
0811 00:12:53.109664 (+ 3015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:53.109696 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:53.110261 (+ 565us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:53.110771 (+ 510us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:53.112205 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:53.114289 (+ 2084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:53.115536 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:53.116539 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:53.116719 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:53.120849 (+ 4130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:53.120861 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:53.120881 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:53.120934 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:53.124706 (+ 3772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:53.125396 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:53.125404 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:53.125412 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:53.125614 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:53.126038 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:53.126042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:53.126046 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:53.126773 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:53.127074 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:53.127425 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 00:12:53.127868 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":403,"threads_started":1}
I20260811 00:12:53.135159 14499 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9402b911714fa680d81711a64c496b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36885 } }
I20260811 00:12:53.135517 14499 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:53.135586 14499 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef9402b911714fa680d81711a64c496b, State: Initialized, Role: FOLLOWER
I20260811 00:12:53.136148 14499 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9402b911714fa680d81711a64c496b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36885 } }
I20260811 00:12:53.136348 14499 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:12:53.136443 14499 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:12:53.136554 14499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:53.139200 14499 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9402b911714fa680d81711a64c496b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36885 } }
I20260811 00:12:53.139631 14499 leader_election.cc:302] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef9402b911714fa680d81711a64c496b; no voters:
I20260811 00:12:53.140188 14499 leader_election.cc:288] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:12:53.140340 14508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:53.141790 14508 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b [term 1 LEADER]: Becoming Leader. State: Replica: ef9402b911714fa680d81711a64c496b, State: Running, Role: LEADER
I20260811 00:12:53.142464 14499 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:12:53.142347 14508 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9402b911714fa680d81711a64c496b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36885 } }
I20260811 00:12:53.145360 14513 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef9402b911714fa680d81711a64c496b. Latest consensus state: current_term: 1 leader_uuid: "ef9402b911714fa680d81711a64c496b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9402b911714fa680d81711a64c496b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36885 } } }
I20260811 00:12:53.145750 14513 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:53.146311 14509 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef9402b911714fa680d81711a64c496b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9402b911714fa680d81711a64c496b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36885 } } }
I20260811 00:12:53.146641 14509 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:53.146749 14526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:12:53.152633 14526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:12:53.163443 14526 catalog_manager.cc:1347] Generated new cluster ID: 1206a13a04524023af7d50797cf74375
I20260811 00:12:53.163565 14526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:12:53.216568 14526 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:12:53.218547 14526 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:12:53.233743 14526 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b: Generated new TSK 0
I20260811 00:12:53.234798 14526 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
I20260811 00:12:53.291040 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:36885
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:42133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:53.627324 14550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:53.627614 14550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:53.627782 14550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:53.634974 14550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:53.635143 14550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:53.635308 14550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:12:53.644896 14550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:42133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:36885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:53.646320 14550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:53.647783 14550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:53.661128 14561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:53.661237 14562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:53.661262 14565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:53.661995 14550 server_base.cc:1034] running on GCE node
I20260811 00:12:53.662593 14550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:53.663410 14550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:53.664625 14550 hybrid_clock.cc:648] HybridClock initialized: now 1786407173664562 us; error 60 us; skew 500 ppm
I20260811 00:12:53.664925 14550 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:53 dist-test-slave-8hhm krb5kdc[14366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407173, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:53.668087 14550 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:12:53.669289 14550 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:53.669420 14550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:53.669739 14550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:53.672477 14550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "e9ef4a7fcb1846ba9e37c0bee65333b8"
format_stamp: "Formatted at 2026-08-11 00:12:53 on dist-test-slave-8hhm"
server_key: "e87cf3550c45934ec503c1e746425e12"
server_key_iv: "2bec7cad9c56956a6142377c07f6f664"
server_key_version: "encryptionkey@0"
I20260811 00:12:53.673341 14550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "e9ef4a7fcb1846ba9e37c0bee65333b8"
format_stamp: "Formatted at 2026-08-11 00:12:53 on dist-test-slave-8hhm"
server_key: "e87cf3550c45934ec503c1e746425e12"
server_key_iv: "2bec7cad9c56956a6142377c07f6f664"
server_key_version: "encryptionkey@0"
I20260811 00:12:53.678144 14550 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 00:12:53.681543 14573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:53.682832 14550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 00:12:53.683022 14550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "e9ef4a7fcb1846ba9e37c0bee65333b8"
format_stamp: "Formatted at 2026-08-11 00:12:53 on dist-test-slave-8hhm"
server_key: "e87cf3550c45934ec503c1e746425e12"
server_key_iv: "2bec7cad9c56956a6142377c07f6f664"
server_key_version: "encryptionkey@0"
I20260811 00:12:53.683267 14550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:53.705534 14550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:53.709340 14550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:53.709616 14550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:53.710489 14550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:53.711733 14550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:53.711879 14550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:53.712024 14550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:53.712121 14550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:53.748080 14550 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:45479
I20260811 00:12:53.748162 14769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:45479 every 8 connection(s)
I20260811 00:12:53.748914 14550 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:12:53.755805 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 14550
I20260811 00:12:53.756026 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:12:53.756834 7544 external_mini_cluster.cc:1428] Setting key c256d97f266fb964ef29ebcd6c687438
Aug 11 00:12:53 dist-test-slave-8hhm krb5kdc[14366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407173, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:53.770681 14773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:53.751374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:55515 (local address 127.7.94.62:36885)
0811 00:12:53.751802 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:53.751806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:53.752799 (+ 993us) server_negotiation.cc:407] Connection header received
0811 00:12:53.754680 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:53.754685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:53.754834 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:53.754950 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:53.757065 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:53.757980 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:53.760053 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:53.760437 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:53.760525 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:53.764713 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:53.764718 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:53.764721 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:53.764751 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:53.767248 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:53.768155 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:53.768160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:53.768164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:53.768311 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:53.768785 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:53.768788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:53.768791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:53.769082 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:53.769255 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:53.769928 (+ 673us) server_negotiation.cc:299] Negotiation successful
0811 00:12:53.770180 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":209,"threads_started":1}
I20260811 00:12:53.771205 14774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:53.751968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36885 (local address 127.7.94.1:55515)
0811 00:12:53.752649 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:53.752687 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:53.754409 (+ 1722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:53.755199 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:53.755207 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:53.756038 (+ 831us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:53.756864 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:53.756891 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:53.758145 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:53.758151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:53.759893 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:53.759910 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:53.760661 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:53.760671 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:53.760999 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:53.761749 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:53.761782 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:53.764493 (+ 2711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:53.767391 (+ 2898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:53.767400 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:53.767414 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:53.768021 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:53.768425 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:53.768430 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:53.768436 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:53.768678 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:53.769249 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:53.769256 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:53.769541 (+ 285us) client_negotiation.cc:769] Sending connection context
0811 00:12:53.769766 (+ 225us) client_negotiation.cc:240] Negotiation successful
0811 00:12:53.770134 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":277,"threads_started":1}
I20260811 00:12:53.772764 14770 heartbeater.cc:347] Connected to a master server at 127.7.94.62:36885
I20260811 00:12:53.773084 14770 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:53.773926 14770 heartbeater.cc:511] Master 127.7.94.62:36885 requested a full tablet report, sending...
I20260811 00:12:53.776243 14441 ts_manager.cc:194] Registered new tserver with Master: e9ef4a7fcb1846ba9e37c0bee65333b8 (127.7.94.1:45479)
I20260811 00:12:53.778460 14441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:55515
WARNING: no policy specified for kudu/127.7.94.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
I20260811 00:12:53.810603 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:36885
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:42133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:54.167075 14777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:54.167339 14777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:54.167536 14777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:54.174563 14777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:54.174661 14777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:54.174855 14777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:12:54.184314 14777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:42133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:36885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:54.185627 14777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:54.186978 14777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:54.199528 14783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:54.199556 14787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:54.199600 14785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:54.200376 14777 server_base.cc:1034] running on GCE node
I20260811 00:12:54.200966 14777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:54.201936 14777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:54.203181 14777 hybrid_clock.cc:648] HybridClock initialized: now 1786407174203126 us; error 59 us; skew 500 ppm
I20260811 00:12:54.203504 14777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:54 dist-test-slave-8hhm krb5kdc[14366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407174, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:54.206643 14777 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:12:54.207726 14777 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:54.207865 14777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:54.208196 14777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:54.211091 14777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "f6fb1bf53f1b4b749818f3a3bd532f7e"
format_stamp: "Formatted at 2026-08-11 00:12:54 on dist-test-slave-8hhm"
server_key: "78aafd8217a78a3799b306e217ccf4ee"
server_key_iv: "11f1864046a0f857326311ecae01d887"
server_key_version: "encryptionkey@0"
I20260811 00:12:54.211926 14777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "f6fb1bf53f1b4b749818f3a3bd532f7e"
format_stamp: "Formatted at 2026-08-11 00:12:54 on dist-test-slave-8hhm"
server_key: "78aafd8217a78a3799b306e217ccf4ee"
server_key_iv: "11f1864046a0f857326311ecae01d887"
server_key_version: "encryptionkey@0"
I20260811 00:12:54.216687 14777 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 00:12:54.220113 14801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:54.221421 14777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:12:54.221606 14777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "f6fb1bf53f1b4b749818f3a3bd532f7e"
format_stamp: "Formatted at 2026-08-11 00:12:54 on dist-test-slave-8hhm"
server_key: "78aafd8217a78a3799b306e217ccf4ee"
server_key_iv: "11f1864046a0f857326311ecae01d887"
server_key_version: "encryptionkey@0"
I20260811 00:12:54.221824 14777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:54.234843 14777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:54.238659 14777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:54.238948 14777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:54.239825 14777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:54.241042 14777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:54.241176 14777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:54.241326 14777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:54.241412 14777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:54.274562 14777 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:38369
I20260811 00:12:54.274646 14996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:38369 every 8 connection(s)
I20260811 00:12:54.275406 14777 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:12:54.282464 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 14777
I20260811 00:12:54.282646 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:12:54.283077 7544 external_mini_cluster.cc:1428] Setting key 5280d7a83d8da01db3992cc83de6dec4
Aug 11 00:12:54 dist-test-slave-8hhm krb5kdc[14366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407174, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:54.296960 15000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:54.277883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:35045 (local address 127.7.94.62:36885)
0811 00:12:54.278320 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:54.278324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:54.279318 (+ 994us) server_negotiation.cc:407] Connection header received
0811 00:12:54.281083 (+ 1765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:54.281088 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:54.281161 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:54.281318 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:54.282842 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.283645 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.285645 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.286099 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.286196 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:54.290709 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:54.290714 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:54.290717 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:54.290765 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:54.293814 (+ 3049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:54.294488 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:54.294496 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:54.294501 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:54.294670 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:54.295049 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:54.295054 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:54.295059 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:54.295442 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:54.295585 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:54.296198 (+ 613us) server_negotiation.cc:299] Negotiation successful
0811 00:12:54.296441 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":238,"threads_started":1}
I20260811 00:12:54.297554 15002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:54.278467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36885 (local address 127.7.94.2:35045)
0811 00:12:54.279164 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:54.279200 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:54.280834 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:54.281494 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:54.281502 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:54.281997 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:54.282646 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:54.282666 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.283829 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.283834 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:54.285492 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:54.285509 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.286338 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.286348 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:54.286706 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:54.287453 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:54.287494 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:54.290482 (+ 2988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:54.293970 (+ 3488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:54.293976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:54.293987 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:54.294376 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:54.294782 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:54.294785 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:54.294788 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:54.294944 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:54.295554 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:54.295559 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:54.295827 (+ 268us) client_negotiation.cc:769] Sending connection context
0811 00:12:54.296069 (+ 242us) client_negotiation.cc:240] Negotiation successful
0811 00:12:54.296426 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":295,"threads_started":1}
I20260811 00:12:54.299113 14997 heartbeater.cc:347] Connected to a master server at 127.7.94.62:36885
I20260811 00:12:54.299460 14997 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:54.300307 14997 heartbeater.cc:511] Master 127.7.94.62:36885 requested a full tablet report, sending...
I20260811 00:12:54.301749 14441 ts_manager.cc:194] Registered new tserver with Master: f6fb1bf53f1b4b749818f3a3bd532f7e (127.7.94.2:38369)
I20260811 00:12:54.302541 14441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:35045
WARNING: no policy specified for kudu/127.7.94.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
I20260811 00:12:54.335904 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:36885
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:42133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:12:54.658501 15006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:54.658793 15006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:54.658957 15006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:54.665704 15006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:54.665856 15006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:54.666031 15006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:12:54.675452 15006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:42133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:36885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:54.676743 15006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:54.678062 15006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:54.690999 15015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:54.690950 15016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:54.690886 15018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:54.691802 15006 server_base.cc:1034] running on GCE node
I20260811 00:12:54.692425 15006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:54.693416 15006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:54.694640 15006 hybrid_clock.cc:648] HybridClock initialized: now 1786407174694606 us; error 48 us; skew 500 ppm
I20260811 00:12:54.694974 15006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:12:54 dist-test-slave-8hhm krb5kdc[14366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407174, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:12:54.698195 15006 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:12:54.699270 15006 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:54.699402 15006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:54.699728 15006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:54.702675 15006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "f7a629ac353e48ad8d9b73086ceae34b"
format_stamp: "Formatted at 2026-08-11 00:12:54 on dist-test-slave-8hhm"
server_key: "ad69ac32d5be5f2768112aa9c44a32a5"
server_key_iv: "fe5c409ff43c3faeef10a1938d0b2727"
server_key_version: "encryptionkey@0"
I20260811 00:12:54.703564 15006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "f7a629ac353e48ad8d9b73086ceae34b"
format_stamp: "Formatted at 2026-08-11 00:12:54 on dist-test-slave-8hhm"
server_key: "ad69ac32d5be5f2768112aa9c44a32a5"
server_key_iv: "fe5c409ff43c3faeef10a1938d0b2727"
server_key_version: "encryptionkey@0"
I20260811 00:12:54.708725 15006 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260811 00:12:54.712142 15030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:54.713411 15006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:12:54.713603 15006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "f7a629ac353e48ad8d9b73086ceae34b"
format_stamp: "Formatted at 2026-08-11 00:12:54 on dist-test-slave-8hhm"
server_key: "ad69ac32d5be5f2768112aa9c44a32a5"
server_key_iv: "fe5c409ff43c3faeef10a1938d0b2727"
server_key_version: "encryptionkey@0"
I20260811 00:12:54.713814 15006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:54.732215 15006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:54.735924 15006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:54.736195 15006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:54.737129 15006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:54.738464 15006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:54.738583 15006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:54.738713 15006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:54.738813 15006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:54.773200 15006 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:37793
I20260811 00:12:54.773314 15220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:37793 every 8 connection(s)
I20260811 00:12:54.774061 15006 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:12:54.776347 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 15006
I20260811 00:12:54.776574 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:12:54.777082 7544 external_mini_cluster.cc:1428] Setting key 87438618ff94750d423b0083ee60188f
I20260811 00:12:54.782006 14770 heartbeater.cc:503] Master 127.7.94.62:36885 was elected leader, sending a full tablet report...
Aug 11 00:12:54 dist-test-slave-8hhm krb5kdc[14366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407174, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:12:54.794377 15000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:54.776588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:36469 (local address 127.7.94.62:36885)
0811 00:12:54.776798 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:54.776804 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:54.778491 (+ 1687us) server_negotiation.cc:407] Connection header received
0811 00:12:54.780530 (+ 2039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:54.780534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:54.780599 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:54.780685 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:54.782282 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.783131 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.784619 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.785027 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.785172 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:54.788583 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:54.788588 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:54.788591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:54.788630 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:54.791147 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:54.791908 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:54.791915 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:54.791923 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:54.792040 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:54.792467 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:54.792472 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:54.792475 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:54.792746 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:54.792921 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:54.793651 (+ 730us) server_negotiation.cc:299] Negotiation successful
0811 00:12:54.793902 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 00:12:54.795317 15230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:54.777070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36885 (local address 127.7.94.3:36469)
0811 00:12:54.778337 (+ 1267us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:54.778381 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:54.780276 (+ 1895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:54.780880 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:54.780887 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:54.781470 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:54.782089 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:54.782107 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.783291 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.783302 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:54.784471 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:54.784483 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.785280 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.785288 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:54.785507 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:54.786183 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:12:54.786206 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:54.788395 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:54.791288 (+ 2893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:54.791296 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:54.791311 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:54.791779 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:54.792161 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:12:54.792166 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:12:54.792170 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:12:54.792347 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:12:54.792900 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:54.792908 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:54.793271 (+ 363us) client_negotiation.cc:769] Sending connection context
0811 00:12:54.793537 (+ 266us) client_negotiation.cc:240] Negotiation successful
0811 00:12:54.794100 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":989,"thread_start_us":775,"threads_started":1}
I20260811 00:12:54.796784 15224 heartbeater.cc:347] Connected to a master server at 127.7.94.62:36885
I20260811 00:12:54.797127 15224 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:54.797797 15224 heartbeater.cc:511] Master 127.7.94.62:36885 requested a full tablet report, sending...
I20260811 00:12:54.799278 14441 ts_manager.cc:194] Registered new tserver with Master: f7a629ac353e48ad8d9b73086ceae34b (127.7.94.3:37793)
I20260811 00:12:54.800040 14441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:36469
I20260811 00:12:54.804436 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:12:54.820926 15000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:54.808881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60816 (local address 127.7.94.62:36885)
0811 00:12:54.809049 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:54.809053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:54.809607 (+ 554us) server_negotiation.cc:407] Connection header received
0811 00:12:54.809733 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:54.809737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:54.809805 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:54.809881 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:54.811243 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.811983 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.813143 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.813585 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.813691 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:54.815791 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:54.815795 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:54.815798 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:54.815828 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:54.818588 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:54.819181 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:54.819184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:54.819186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:54.819295 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:54.819608 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:54.819611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:54.819613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:54.819908 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:54.820105 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:54.820376 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 00:12:54.820524 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 00:12:54.825126 14441 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:12:54.831974 14441 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:12:54.855696 15250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:54.843883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:38369 (local address 127.0.0.1:42258)
0811 00:12:54.845244 (+ 1361us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:54.845300 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:54.845501 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:54.846463 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:54.846475 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:54.846519 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:54.847088 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:54.847116 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.850261 (+ 3145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.850269 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:54.852462 (+ 2193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:54.852473 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.854653 (+ 2180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.854660 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:54.854840 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:54.854898 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 00:12:54.855033 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 00:12:54.855236 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1078,"mutex_wait_us":60,"spinlock_wait_cycles":5760,"thread_start_us":1311,"threads_started":1}
I20260811 00:12:54.855844 15252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:54.844153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:37793 (local address 127.0.0.1:43028)
0811 00:12:54.845244 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:54.845300 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:54.845501 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:54.847745 (+ 2244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:54.847750 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:54.847773 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:54.848267 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:54.848282 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.850344 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.850349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:54.852735 (+ 2386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:54.852749 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.854902 (+ 2153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.854908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:54.855081 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:54.855122 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 00:12:54.855208 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 00:12:54.855338 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"mutex_wait_us":24,"spinlock_wait_cycles":10112,"thread_start_us":941,"threads_started":1}
I20260811 00:12:54.856967 15254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:54.844809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43028 (local address 127.7.94.3:37793)
0811 00:12:54.847089 (+ 2280us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:54.847101 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:54.847143 (+ 42us) server_negotiation.cc:407] Connection header received
0811 00:12:54.847274 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:54.847285 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:54.847559 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:54.847753 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:54.848733 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.850227 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.852944 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.854389 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.854502 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:54.854718 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:54.856399 (+ 1681us) server_negotiation.cc:299] Negotiation successful
0811 00:12:54.856563 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1162,"thread_start_us":1302,"threads_started":1}
I20260811 00:12:54.856994 15251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:54.844194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42258 (local address 127.7.94.2:38369)
0811 00:12:54.844786 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:54.844797 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:54.845699 (+ 902us) server_negotiation.cc:407] Connection header received
0811 00:12:54.845830 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:54.845839 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:54.846196 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:54.846396 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:54.848658 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.850129 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.852710 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.854190 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.854325 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:54.854523 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:54.856058 (+ 1535us) server_negotiation.cc:299] Negotiation successful
0811 00:12:54.856360 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":290,"threads_started":1}
I20260811 00:12:54.857286 15255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:54.845951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:45479 (local address 127.0.0.1:41998)
0811 00:12:54.848914 (+ 2963us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:54.848932 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:54.849091 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:54.849666 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:54.849671 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:54.849695 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:54.850011 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:54.850021 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.851945 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.851950 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:54.854111 (+ 2161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:54.854126 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.856462 (+ 2336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.856468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:54.856649 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:54.856683 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 00:12:54.856770 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 00:12:54.856892 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2822,"thread_start_us":2657,"threads_started":1}
I20260811 00:12:54.858031 15256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:54.846225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41998 (local address 127.7.94.1:45479)
0811 00:12:54.848474 (+ 2249us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:54.848482 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:54.849077 (+ 595us) server_negotiation.cc:407] Connection header received
0811 00:12:54.849267 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:54.849275 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:54.849474 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:54.849671 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:54.850168 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.851783 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.854299 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:54.855777 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:54.855978 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:54.856212 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:54.857547 (+ 1335us) server_negotiation.cc:299] Negotiation successful
0811 00:12:54.857714 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2051,"thread_start_us":2198,"threads_started":1}
I20260811 00:12:54.860334 14884 tablet_service.cc:1467] Processing CreateTablet for tablet 9ae1ee2b87f14d94ae502d4aeb4e226a (DEFAULT_TABLE table=test-table [id=105e1747fd2f49ee8d2eaa68a1a79365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:54.860344 15110 tablet_service.cc:1467] Processing CreateTablet for tablet 9ae1ee2b87f14d94ae502d4aeb4e226a (DEFAULT_TABLE table=test-table [id=105e1747fd2f49ee8d2eaa68a1a79365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:54.861521 14884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ae1ee2b87f14d94ae502d4aeb4e226a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:54.861527 15110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ae1ee2b87f14d94ae502d4aeb4e226a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:54.864564 14652 tablet_service.cc:1467] Processing CreateTablet for tablet 9ae1ee2b87f14d94ae502d4aeb4e226a (DEFAULT_TABLE table=test-table [id=105e1747fd2f49ee8d2eaa68a1a79365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:54.865882 14652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ae1ee2b87f14d94ae502d4aeb4e226a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:54.872784 15259 tablet_bootstrap.cc:492] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e: Bootstrap starting.
I20260811 00:12:54.876322 15259 tablet_bootstrap.cc:654] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:54.876417 15262 tablet_bootstrap.cc:492] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b: Bootstrap starting.
I20260811 00:12:54.877727 15259 log.cc:824] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:54.879835 15262 tablet_bootstrap.cc:654] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:54.879832 15265 tablet_bootstrap.cc:492] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8: Bootstrap starting.
I20260811 00:12:54.880885 15259 tablet_bootstrap.cc:492] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e: No bootstrap required, opened a new log
I20260811 00:12:54.881234 15259 ts_tablet_manager.cc:1397] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 00:12:54.881213 15262 log.cc:824] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:54.883320 15265 tablet_bootstrap.cc:654] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:54.884573 15265 log.cc:824] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:54.891073 15262 tablet_bootstrap.cc:492] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b: No bootstrap required, opened a new log
I20260811 00:12:54.891577 15262 ts_tablet_manager.cc:1397] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.005s
I20260811 00:12:54.893441 15265 tablet_bootstrap.cc:492] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8: No bootstrap required, opened a new log
I20260811 00:12:54.893108 15259 raft_consensus.cc:348] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6fb1bf53f1b4b749818f3a3bd532f7e" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38369 } } peers { permanent_uuid: "f7a629ac353e48ad8d9b73086ceae34b" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37793 } } peers { permanent_uuid: "e9ef4a7fcb1846ba9e37c0bee65333b8" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 45479 } }
I20260811 00:12:54.893580 15259 raft_consensus.cc:374] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:54.893709 15259 raft_consensus.cc:729] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6fb1bf53f1b4b749818f3a3bd532f7e, State: Initialized, Role: FOLLOWER
I20260811 00:12:54.893748 15265 ts_tablet_manager.cc:1397] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.008s
I20260811 00:12:54.894373 15259 consensus_queue.cc:260] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6fb1bf53f1b4b749818f3a3bd532f7e" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38369 } } peers { permanent_uuid: "f7a629ac353e48ad8d9b73086ceae34b" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37793 } } peers { permanent_uuid: "e9ef4a7fcb1846ba9e37c0bee65333b8" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 45479 } }
I20260811 00:12:54.895560 14997 heartbeater.cc:503] Master 127.7.94.62:36885 was elected leader, sending a full tablet report...
I20260811 00:12:54.896163 15259 ts_tablet_manager.cc:1428] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e: Time spent starting tablet: real 0.015s user 0.010s sys 0.003s
I20260811 00:12:54.903795 15262 raft_consensus.cc:348] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6fb1bf53f1b4b749818f3a3bd532f7e" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38369 } } peers { permanent_uuid: "f7a629ac353e48ad8d9b73086ceae34b" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37793 } } peers { permanent_uuid: "e9ef4a7fcb1846ba9e37c0bee65333b8" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 45479 } }
I20260811 00:12:54.904250 15262 raft_consensus.cc:374] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:54.904027 15265 raft_consensus.cc:348] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6fb1bf53f1b4b749818f3a3bd532f7e" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38369 } } peers { permanent_uuid: "f7a629ac353e48ad8d9b73086ceae34b" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37793 } } peers { permanent_uuid: "e9ef4a7fcb1846ba9e37c0bee65333b8" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 45479 } }
I20260811 00:12:54.904354 15262 raft_consensus.cc:729] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7a629ac353e48ad8d9b73086ceae34b, State: Initialized, Role: FOLLOWER
I20260811 00:12:54.904453 15265 raft_consensus.cc:374] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:54.904554 15265 raft_consensus.cc:729] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9ef4a7fcb1846ba9e37c0bee65333b8, State: Initialized, Role: FOLLOWER
I20260811 00:12:54.905045 15262 consensus_queue.cc:260] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6fb1bf53f1b4b749818f3a3bd532f7e" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38369 } } peers { permanent_uuid: "f7a629ac353e48ad8d9b73086ceae34b" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37793 } } peers { permanent_uuid: "e9ef4a7fcb1846ba9e37c0bee65333b8" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 45479 } }
I20260811 00:12:54.905079 15265 consensus_queue.cc:260] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6fb1bf53f1b4b749818f3a3bd532f7e" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38369 } } peers { permanent_uuid: "f7a629ac353e48ad8d9b73086ceae34b" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37793 } } peers { permanent_uuid: "e9ef4a7fcb1846ba9e37c0bee65333b8" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 45479 } }
I20260811 00:12:54.906186 15224 heartbeater.cc:503] Master 127.7.94.62:36885 was elected leader, sending a full tablet report...
I20260811 00:12:54.907383 15265 ts_tablet_manager.cc:1428] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260811 00:12:54.907727 15262 ts_tablet_manager.cc:1428] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b: Time spent starting tablet: real 0.016s user 0.013s sys 0.001s
W20260811 00:12:55.001475 14771 tablet.cc:2369] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:12:55.025609 15226 tablet.cc:2369] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:12:55.027208 14999 tablet.cc:2369] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:12:55.084956 15270 raft_consensus.cc:482] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:12:55.085391 15270 raft_consensus.cc:504] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6fb1bf53f1b4b749818f3a3bd532f7e" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38369 } } peers { permanent_uuid: "f7a629ac353e48ad8d9b73086ceae34b" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37793 } } peers { permanent_uuid: "e9ef4a7fcb1846ba9e37c0bee65333b8" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 45479 } }
I20260811 00:12:55.087282 15270 leader_election.cc:288] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6fb1bf53f1b4b749818f3a3bd532f7e (127.7.94.2:38369), e9ef4a7fcb1846ba9e37c0bee65333b8 (127.7.94.1:45479)
I20260811 00:12:55.093799 15251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:55.087626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:38473 (local address 127.7.94.2:38369)
0811 00:12:55.087810 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:55.087817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:55.087939 (+ 122us) server_negotiation.cc:407] Connection header received
0811 00:12:55.088193 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:55.088197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:55.088261 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:55.088363 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:55.088815 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:55.089872 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:55.091786 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:55.092852 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:55.092966 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:55.093033 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:55.093449 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 00:12:55.093541 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 00:12:55.093885 15230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:55.087478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:38369 (local address 127.7.94.3:38473)
0811 00:12:55.087809 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:55.087832 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:55.088021 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:55.088387 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:55.088396 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:55.088422 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:55.088649 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:55.088662 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:55.090018 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:55.090024 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:55.091598 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:55.091610 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:55.093026 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:55.093032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:55.093280 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:55.093319 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 00:12:55.093420 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 00:12:55.093555 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":21}
I20260811 00:12:55.094525 15256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:55.087822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:54545 (local address 127.7.94.1:45479)
0811 00:12:55.088017 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:55.088024 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:55.088644 (+ 620us) server_negotiation.cc:407] Connection header received
0811 00:12:55.088767 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:55.088776 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:55.088849 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:55.088927 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:12:55.089574 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:55.090534 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:55.092531 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:55.093583 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:55.093712 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:55.093820 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:55.094202 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 00:12:55.094332 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 00:12:55.094689 15273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:55.087692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:45479 (local address 127.7.94.3:54545)
0811 00:12:55.088536 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:55.088550 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:55.088661 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:55.088986 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:55.088992 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:55.089017 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:12:55.089407 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:55.089419 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:55.090732 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:55.090736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:55.092365 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:55.092381 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:55.093801 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:55.093820 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:55.093999 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:55.094040 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 00:12:55.094142 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 00:12:55.094297 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"mutex_wait_us":1,"thread_start_us":509,"threads_started":1}
I20260811 00:12:55.095098 14918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae1ee2b87f14d94ae502d4aeb4e226a" candidate_uuid: "f7a629ac353e48ad8d9b73086ceae34b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6fb1bf53f1b4b749818f3a3bd532f7e" is_pre_election: true
I20260811 00:12:55.095710 14918 raft_consensus.cc:2454] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7a629ac353e48ad8d9b73086ceae34b in term 0.
I20260811 00:12:55.095726 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae1ee2b87f14d94ae502d4aeb4e226a" candidate_uuid: "f7a629ac353e48ad8d9b73086ceae34b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9ef4a7fcb1846ba9e37c0bee65333b8" is_pre_election: true
I20260811 00:12:55.096254 14687 raft_consensus.cc:2454] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7a629ac353e48ad8d9b73086ceae34b in term 0.
I20260811 00:12:55.096419 15034 leader_election.cc:302] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f6fb1bf53f1b4b749818f3a3bd532f7e, f7a629ac353e48ad8d9b73086ceae34b; no voters:
I20260811 00:12:55.096869 15270 raft_consensus.cc:2781] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:12:55.097014 15270 raft_consensus.cc:482] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:12:55.097133 15270 raft_consensus.cc:3037] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:55.100278 15270 raft_consensus.cc:504] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6fb1bf53f1b4b749818f3a3bd532f7e" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38369 } } peers { permanent_uuid: "f7a629ac353e48ad8d9b73086ceae34b" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37793 } } peers { permanent_uuid: "e9ef4a7fcb1846ba9e37c0bee65333b8" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 45479 } }
I20260811 00:12:55.100929 15270 leader_election.cc:288] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [CANDIDATE]: Term 1 election: Requested vote from peers f6fb1bf53f1b4b749818f3a3bd532f7e (127.7.94.2:38369), e9ef4a7fcb1846ba9e37c0bee65333b8 (127.7.94.1:45479)
I20260811 00:12:55.101423 14918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae1ee2b87f14d94ae502d4aeb4e226a" candidate_uuid: "f7a629ac353e48ad8d9b73086ceae34b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6fb1bf53f1b4b749818f3a3bd532f7e"
I20260811 00:12:55.101493 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae1ee2b87f14d94ae502d4aeb4e226a" candidate_uuid: "f7a629ac353e48ad8d9b73086ceae34b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9ef4a7fcb1846ba9e37c0bee65333b8"
I20260811 00:12:55.101652 14918 raft_consensus.cc:3037] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:55.101696 14687 raft_consensus.cc:3037] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:55.104913 14687 raft_consensus.cc:2454] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7a629ac353e48ad8d9b73086ceae34b in term 1.
I20260811 00:12:55.104913 14918 raft_consensus.cc:2454] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7a629ac353e48ad8d9b73086ceae34b in term 1.
I20260811 00:12:55.105340 15034 leader_election.cc:302] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f6fb1bf53f1b4b749818f3a3bd532f7e, f7a629ac353e48ad8d9b73086ceae34b; no voters:
I20260811 00:12:55.105677 15270 raft_consensus.cc:2781] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:55.106247 15270 raft_consensus.cc:686] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [term 1 LEADER]: Becoming Leader. State: Replica: f7a629ac353e48ad8d9b73086ceae34b, State: Running, Role: LEADER
I20260811 00:12:55.106781 15270 consensus_queue.cc:237] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6fb1bf53f1b4b749818f3a3bd532f7e" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38369 } } peers { permanent_uuid: "f7a629ac353e48ad8d9b73086ceae34b" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37793 } } peers { permanent_uuid: "e9ef4a7fcb1846ba9e37c0bee65333b8" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 45479 } }
I20260811 00:12:55.111444 14438 catalog_manager.cc:5495] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b reported cstate change: term changed from 0 to 1, leader changed from <none> to f7a629ac353e48ad8d9b73086ceae34b (127.7.94.3). New cstate: current_term: 1 leader_uuid: "f7a629ac353e48ad8d9b73086ceae34b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6fb1bf53f1b4b749818f3a3bd532f7e" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7a629ac353e48ad8d9b73086ceae34b" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 37793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9ef4a7fcb1846ba9e37c0bee65333b8" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 45479 } health_report { overall_health: UNKNOWN } } }
I20260811 00:12:55.150797 15254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:55.145609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43044 (local address 127.7.94.3:37793)
0811 00:12:55.145843 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:55.145848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:55.145966 (+ 118us) server_negotiation.cc:407] Connection header received
0811 00:12:55.146167 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:55.146171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:55.146260 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:55.146389 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:12:55.146765 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:55.147666 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:55.148918 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:55.149317 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:55.149402 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:55.149845 (+ 443us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:12:55.150341 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:12:55.150449 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:55.150529 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 00:12:55.150590 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 00:12:55.158936 14687 raft_consensus.cc:1261] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8 [term 1 FOLLOWER]: Refusing update from remote peer f7a629ac353e48ad8d9b73086ceae34b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:12:55.158936 14918 raft_consensus.cc:1261] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e [term 1 FOLLOWER]: Refusing update from remote peer f7a629ac353e48ad8d9b73086ceae34b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:12:55.159967 15275 consensus_queue.cc:1035] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9ef4a7fcb1846ba9e37c0bee65333b8" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 45479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:12:55.160331 15270 consensus_queue.cc:1035] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6fb1bf53f1b4b749818f3a3bd532f7e" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:12:55.172058 15279 mvcc.cc:205] Tried to move back new op lower bound from 7317123789440913408 to 7317123789241884672. Current Snapshot: MvccSnapshot[applied={T|T < 7317123789440913408}]
I20260811 00:12:55.173377 15280 mvcc.cc:205] Tried to move back new op lower bound from 7317123789440913408 to 7317123789241884672. Current Snapshot: MvccSnapshot[applied={T|T < 7317123789440913408}]
I20260811 00:12:55.185423 14440 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:12:55.185755 14440 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60816:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:12:55.189654 14440 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef9402b911714fa680d81711a64c496b: Sending DeleteTablet for 3 replicas of tablet 9ae1ee2b87f14d94ae502d4aeb4e226a
I20260811 00:12:55.191020 15110 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ae1ee2b87f14d94ae502d4aeb4e226a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:55 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:43028
I20260811 00:12:55.191033 14884 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ae1ee2b87f14d94ae502d4aeb4e226a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:55 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:42258
I20260811 00:12:55.191112 14652 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ae1ee2b87f14d94ae502d4aeb4e226a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:55 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:41998
I20260811 00:12:55.192364 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 14550
I20260811 00:12:55.204150 15296 tablet_replica.cc:330] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b: stopping tablet replica
I20260811 00:12:55.204754 15296 raft_consensus.cc:2229] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:12:55.207388 15299 tablet_replica.cc:330] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e: stopping tablet replica
I20260811 00:12:55.208081 15296 raft_consensus.cc:2258] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:55.208778 15299 raft_consensus.cc:2229] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:12:55.211812 15296 ts_tablet_manager.cc:1905] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:55.212039 15299 raft_consensus.cc:2258] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:55.214396 15299 ts_tablet_manager.cc:1905] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:55.215020 15298 tablet_replica.cc:330] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8: stopping tablet replica
I20260811 00:12:55.215458 15298 raft_consensus.cc:2229] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:12:55.215965 15298 raft_consensus.cc:2258] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:55.218199 15298 ts_tablet_manager.cc:1905] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:55.219637 15296 ts_tablet_manager.cc:1918] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:12:55.219864 15296 log.cc:1196] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/9ae1ee2b87f14d94ae502d4aeb4e226a
I20260811 00:12:55.220435 15296 ts_tablet_manager.cc:1939] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f7a629ac353e48ad8d9b73086ceae34b: Deleting consensus metadata
Aug 11 00:12:55 dist-test-slave-8hhm krb5kdc[14366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:12:55.222388 14423 catalog_manager.cc:4841] TS f7a629ac353e48ad8d9b73086ceae34b (127.7.94.3:37793): tablet 9ae1ee2b87f14d94ae502d4aeb4e226a (table test-table [id=105e1747fd2f49ee8d2eaa68a1a79365]) successfully deleted
I20260811 00:12:55.223979 15299 ts_tablet_manager.cc:1918] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:12:55.224239 15299 log.cc:1196] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/9ae1ee2b87f14d94ae502d4aeb4e226a
I20260811 00:12:55.224814 15299 ts_tablet_manager.cc:1939] T 9ae1ee2b87f14d94ae502d4aeb4e226a P f6fb1bf53f1b4b749818f3a3bd532f7e: Deleting consensus metadata
I20260811 00:12:55.226358 15298 ts_tablet_manager.cc:1918] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:12:55.226586 15298 log.cc:1196] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/9ae1ee2b87f14d94ae502d4aeb4e226a
I20260811 00:12:55.227209 15298 ts_tablet_manager.cc:1939] T 9ae1ee2b87f14d94ae502d4aeb4e226a P e9ef4a7fcb1846ba9e37c0bee65333b8: Deleting consensus metadata
I20260811 00:12:55.227236 14419 catalog_manager.cc:4841] TS f6fb1bf53f1b4b749818f3a3bd532f7e (127.7.94.2:38369): tablet 9ae1ee2b87f14d94ae502d4aeb4e226a (table test-table [id=105e1747fd2f49ee8d2eaa68a1a79365]) successfully deleted
I20260811 00:12:55.228983 14423 catalog_manager.cc:4841] TS e9ef4a7fcb1846ba9e37c0bee65333b8 (127.7.94.1:45479): tablet 9ae1ee2b87f14d94ae502d4aeb4e226a (table test-table [id=105e1747fd2f49ee8d2eaa68a1a79365]) successfully deleted
I20260811 00:12:55.229717 15256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:55.194092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42014 (local address 127.7.94.1:45479)
0811 00:12:55.194334 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:55.194343 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:55.204268 (+ 9925us) server_negotiation.cc:407] Connection header received
0811 00:12:55.204562 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:55.204569 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:55.204671 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:55.204751 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:55.206648 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:55.207911 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:55.209824 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:55.210517 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:55.210628 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:55.222080 (+ 11452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:55.222095 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:55.222115 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:55.222169 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:55.226451 (+ 4282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:55.227473 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:55.227484 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:55.227486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:55.227680 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:55.228095 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:55.228100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:55.228105 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:55.228505 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:55.228873 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:55.229120 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 00:12:55.229380 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 00:12:55.230443 14760 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:55.318441 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 14550
I20260811 00:12:55.351327 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 14777
Aug 11 00:12:55 dist-test-slave-8hhm krb5kdc[14366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.2@KRBTEST.COM
I20260811 00:12:55.368114 15251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:55.352185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42270 (local address 127.7.94.2:38369)
0811 00:12:55.352382 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:55.352388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:55.352411 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:12:55.352532 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:55.352538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:55.352631 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:55.352737 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:55.353986 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:55.355143 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:55.356490 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:55.357107 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:55.357366 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:55.361595 (+ 4229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:55.361613 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:55.361635 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:55.361675 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:55.365389 (+ 3714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:55.366004 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:55.366014 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:55.366017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:55.366182 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:55.366520 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:55.366525 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:55.366529 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:55.367021 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:55.367330 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:55.367520 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 00:12:55.367741 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 00:12:55.368788 14989 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:55.458523 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 14777
I20260811 00:12:55.485373 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 15006
Aug 11 00:12:55 dist-test-slave-8hhm krb5kdc[14366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.3@KRBTEST.COM
I20260811 00:12:55.503652 15254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:55.486242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43054 (local address 127.7.94.3:37793)
0811 00:12:55.486423 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:55.486431 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:55.486456 (+ 25us) server_negotiation.cc:407] Connection header received
0811 00:12:55.486644 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:55.486650 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:55.486756 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:55.486844 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:55.488719 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:55.489838 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:55.491590 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:55.492054 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:55.492148 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:55.497762 (+ 5614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:55.497774 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:55.497789 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:12:55.497831 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:55.500846 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:55.501686 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:55.501693 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:55.501695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:55.501835 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:12:55.502255 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:12:55.502258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:12:55.502260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:12:55.502575 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:55.502808 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:55.503161 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 00:12:55.503311 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 00:12:55.504383 15212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:55.623733 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 15006
I20260811 00:12:55.650496 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 14387
I20260811 00:12:55.651626 14490 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:55.760980 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 14387
2026-08-11T00:12:55Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4655 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T00:12:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:12:55Z Disabled control of system clock
I20260811 00:12:55.850196 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:45027
--webserver_interface=127.7.94.62
--webserver_port=0
--builtin_ntp_servers=127.7.94.20:41249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:45027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 00:12:56.176203 15311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:56.176476 15311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:56.176586 15311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:56.183743 15311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:12:56.183864 15311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:56.183904 15311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:56.183988 15311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:12:56.184069 15311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:12:56.193499 15311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:45027
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:45027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:56.194730 15311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:56.196489 15311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:56.208832 15325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:56.208900 15320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:56.208904 15321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:56.210232 15311 server_base.cc:1034] running on GCE node
I20260811 00:12:56.211097 15311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:56.212695 15311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:56.213951 15311 hybrid_clock.cc:648] HybridClock initialized: now 1786407176213880 us; error 52 us; skew 500 ppm
I20260811 00:12:56.214341 15311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:12:56.216696 15311 webserver.cc:460] Webserver started at http://127.7.94.62:37129/ using document root <none> and password file <none>
I20260811 00:12:56.217561 15311 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:56.217720 15311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:56.218210 15311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:56.221557 15311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "ab7f0241ba1f4a55a197412ca12d690b"
format_stamp: "Formatted at 2026-08-11 00:12:56 on dist-test-slave-8hhm"
server_key: "66d5f30066e425799a5462c76e848573"
server_key_iv: "06b2f425c92d1d8bc07d2b75355aa2c3"
server_key_version: "encryptionkey@0"
I20260811 00:12:56.222473 15311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "ab7f0241ba1f4a55a197412ca12d690b"
format_stamp: "Formatted at 2026-08-11 00:12:56 on dist-test-slave-8hhm"
server_key: "66d5f30066e425799a5462c76e848573"
server_key_iv: "06b2f425c92d1d8bc07d2b75355aa2c3"
server_key_version: "encryptionkey@0"
I20260811 00:12:56.227922 15311 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:12:56.231328 15335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:56.232542 15311 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 00:12:56.232729 15311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "ab7f0241ba1f4a55a197412ca12d690b"
format_stamp: "Formatted at 2026-08-11 00:12:56 on dist-test-slave-8hhm"
server_key: "66d5f30066e425799a5462c76e848573"
server_key_iv: "06b2f425c92d1d8bc07d2b75355aa2c3"
server_key_version: "encryptionkey@0"
I20260811 00:12:56.232966 15311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:56.238526 15311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:56.239275 15311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:56.239593 15311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:56.257928 15311 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:45027
I20260811 00:12:56.257951 15415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:45027 every 8 connection(s)
I20260811 00:12:56.259317 15311 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:12:56.260212 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 15311
I20260811 00:12:56.260461 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:12:56.261121 7544 external_mini_cluster.cc:1428] Setting key 4cffd92a4cce0f53b07e48ed44aeaf59
I20260811 00:12:56.263744 15416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:56.269601 15427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:56.263292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59436 (local address 127.7.94.62:45027)
0811 00:12:56.264333 (+ 1041us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:56.264344 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:56.264386 (+ 42us) server_negotiation.cc:407] Connection header received
0811 00:12:56.266035 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:56.266057 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:56.266460 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:56.266783 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:56.267921 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:56.267926 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:56.267938 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:12:56.267952 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:56.268023 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:12:56.268039 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:56.268222 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:56.268341 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 00:12:56.268636 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":407,"threads_started":1}
I20260811 00:12:56.275519 15416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b: Bootstrap starting.
I20260811 00:12:56.278522 15416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:56.279476 15416 log.cc:824] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:56.282305 15416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b: No bootstrap required, opened a new log
I20260811 00:12:56.289403 15416 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab7f0241ba1f4a55a197412ca12d690b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45027 } }
I20260811 00:12:56.289750 15416 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:56.289856 15416 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab7f0241ba1f4a55a197412ca12d690b, State: Initialized, Role: FOLLOWER
I20260811 00:12:56.290429 15416 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab7f0241ba1f4a55a197412ca12d690b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45027 } }
I20260811 00:12:56.290665 15416 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:12:56.290802 15416 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:12:56.290977 15416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:56.292996 15416 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab7f0241ba1f4a55a197412ca12d690b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45027 } }
I20260811 00:12:56.293505 15416 leader_election.cc:302] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab7f0241ba1f4a55a197412ca12d690b; no voters:
I20260811 00:12:56.294064 15416 leader_election.cc:288] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:12:56.294202 15431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:56.294927 15431 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b [term 1 LEADER]: Becoming Leader. State: Replica: ab7f0241ba1f4a55a197412ca12d690b, State: Running, Role: LEADER
I20260811 00:12:56.295480 15431 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab7f0241ba1f4a55a197412ca12d690b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45027 } }
I20260811 00:12:56.296146 15416 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:12:56.298949 15432 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab7f0241ba1f4a55a197412ca12d690b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab7f0241ba1f4a55a197412ca12d690b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45027 } } }
I20260811 00:12:56.299827 15432 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:56.299104 15433 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab7f0241ba1f4a55a197412ca12d690b. Latest consensus state: current_term: 1 leader_uuid: "ab7f0241ba1f4a55a197412ca12d690b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab7f0241ba1f4a55a197412ca12d690b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45027 } } }
I20260811 00:12:56.300351 15433 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:56.301664 15445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:12:56.307384 15445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:12:56.316687 15445 catalog_manager.cc:1347] Generated new cluster ID: 9dd127b1f3ed4458b79767ef625fa24a
I20260811 00:12:56.316844 15445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:12:56.346702 15445 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:12:56.347141 15445 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:12:56.353614 15445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b: Generated new TSK 0
I20260811 00:12:56.354481 15445 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:12:56.371413 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45027
--builtin_ntp_servers=127.7.94.20:41249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 00:12:56.704041 15461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:56.704308 15461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:56.704406 15461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:56.711786 15461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:56.711895 15461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:56.712049 15461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:12:56.720893 15461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:56.722246 15461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:56.724073 15461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:56.737191 15468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:56.737397 15470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:56.737583 15467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:56.738180 15461 server_base.cc:1034] running on GCE node
I20260811 00:12:56.738694 15461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:56.739543 15461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:56.740739 15461 hybrid_clock.cc:648] HybridClock initialized: now 1786407176740690 us; error 43 us; skew 500 ppm
I20260811 00:12:56.741043 15461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:12:56.743233 15461 webserver.cc:460] Webserver started at http://127.7.94.1:44047/ using document root <none> and password file <none>
I20260811 00:12:56.743928 15461 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:56.744052 15461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:56.744376 15461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:56.747263 15461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "ed957c7e078d47b89b841a55acbd7b70"
format_stamp: "Formatted at 2026-08-11 00:12:56 on dist-test-slave-8hhm"
server_key: "6d659f51bdbe6b3b74762b1933995de9"
server_key_iv: "37e71852cff89e3b5bba35580d4e1192"
server_key_version: "encryptionkey@0"
I20260811 00:12:56.748034 15461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "ed957c7e078d47b89b841a55acbd7b70"
format_stamp: "Formatted at 2026-08-11 00:12:56 on dist-test-slave-8hhm"
server_key: "6d659f51bdbe6b3b74762b1933995de9"
server_key_iv: "37e71852cff89e3b5bba35580d4e1192"
server_key_version: "encryptionkey@0"
I20260811 00:12:56.752997 15461 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 00:12:56.756335 15483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:56.757714 15461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:12:56.757889 15461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "ed957c7e078d47b89b841a55acbd7b70"
format_stamp: "Formatted at 2026-08-11 00:12:56 on dist-test-slave-8hhm"
server_key: "6d659f51bdbe6b3b74762b1933995de9"
server_key_iv: "37e71852cff89e3b5bba35580d4e1192"
server_key_version: "encryptionkey@0"
I20260811 00:12:56.758136 15461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:56.763688 15461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:56.764458 15461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:56.764731 15461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:56.765702 15461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:56.767030 15461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:56.767174 15461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:56.767298 15461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:56.767386 15461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:56.800731 15461 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:38799
I20260811 00:12:56.800868 15658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:38799 every 8 connection(s)
I20260811 00:12:56.802232 15461 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:12:56.804579 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 15461
I20260811 00:12:56.804760 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:12:56.805866 7544 external_mini_cluster.cc:1428] Setting key 474fb57b979441115e5c013319b377c3
I20260811 00:12:56.809183 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45027
--builtin_ntp_servers=127.7.94.20:41249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 00:12:56.811177 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:56.804988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:57581 (local address 127.7.94.62:45027)
0811 00:12:56.806925 (+ 1937us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:56.806931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:56.806951 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:12:56.808846 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:56.808851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:56.808896 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:56.808985 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:56.810509 (+ 1524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:56.810513 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:56.810516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:12:56.810539 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:56.810580 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:12:56.810592 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:56.810682 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:56.810918 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 00:12:56.810967 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1814,"thread_start_us":2105,"threads_started":1}
I20260811 00:12:56.812481 15663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:56.805580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:45027 (local address 127.7.94.1:57581)
0811 00:12:56.806775 (+ 1195us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:56.806820 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:56.808553 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:56.809293 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:56.809304 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:56.809436 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:56.810143 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:12:56.810149 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:56.810204 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:12:56.810322 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:56.810688 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:56.810702 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:56.810713 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 00:12:56.810992 (+ 279us) client_negotiation.cc:240] Negotiation successful
0811 00:12:56.811244 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":460,"threads_started":1}
I20260811 00:12:56.814081 15659 heartbeater.cc:347] Connected to a master server at 127.7.94.62:45027
I20260811 00:12:56.814332 15659 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:56.814999 15659 heartbeater.cc:511] Master 127.7.94.62:45027 requested a full tablet report, sending...
I20260811 00:12:56.817478 15357 ts_manager.cc:194] Registered new tserver with Master: ed957c7e078d47b89b841a55acbd7b70 (127.7.94.1:38799)
W20260811 00:12:57.140022 15665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:57.140312 15665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:57.140486 15665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:57.147614 15665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:57.147706 15665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:57.147902 15665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:12:57.156692 15665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:57.157963 15665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:57.159690 15665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:57.172739 15676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:57.172828 15683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:57.172849 15673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:57.173702 15665 server_base.cc:1034] running on GCE node
I20260811 00:12:57.174324 15665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:57.175168 15665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:57.176390 15665 hybrid_clock.cc:648] HybridClock initialized: now 1786407177176354 us; error 50 us; skew 500 ppm
I20260811 00:12:57.176694 15665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:12:57.178956 15665 webserver.cc:460] Webserver started at http://127.7.94.2:34541/ using document root <none> and password file <none>
I20260811 00:12:57.179622 15665 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:57.179750 15665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:57.180052 15665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:57.182928 15665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "7313d798516c4c3b980b4d9fa2c492d3"
format_stamp: "Formatted at 2026-08-11 00:12:57 on dist-test-slave-8hhm"
server_key: "8ed807f0fec0491bd34324aef8ddf36d"
server_key_iv: "761d21a6339b6e28367097aa6424e05c"
server_key_version: "encryptionkey@0"
I20260811 00:12:57.183768 15665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "7313d798516c4c3b980b4d9fa2c492d3"
format_stamp: "Formatted at 2026-08-11 00:12:57 on dist-test-slave-8hhm"
server_key: "8ed807f0fec0491bd34324aef8ddf36d"
server_key_iv: "761d21a6339b6e28367097aa6424e05c"
server_key_version: "encryptionkey@0"
I20260811 00:12:57.188817 15665 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 00:12:57.192040 15692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:57.193333 15665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:12:57.193496 15665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "7313d798516c4c3b980b4d9fa2c492d3"
format_stamp: "Formatted at 2026-08-11 00:12:57 on dist-test-slave-8hhm"
server_key: "8ed807f0fec0491bd34324aef8ddf36d"
server_key_iv: "761d21a6339b6e28367097aa6424e05c"
server_key_version: "encryptionkey@0"
I20260811 00:12:57.193727 15665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:57.199245 15665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:57.199966 15665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:57.200227 15665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:57.201071 15665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:57.202493 15665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:57.202610 15665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:57.202734 15665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:57.202817 15665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:57.236147 15665 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:33153
I20260811 00:12:57.236195 15878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:33153 every 8 connection(s)
I20260811 00:12:57.237692 15665 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:12:57.243330 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 15665
I20260811 00:12:57.243527 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:12:57.244006 7544 external_mini_cluster.cc:1428] Setting key a4f22ddad4ea6331f9690e84d2f7d947
I20260811 00:12:57.245882 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:57.240264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:57183 (local address 127.7.94.62:45027)
0811 00:12:57.240491 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:57.240497 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:57.241899 (+ 1402us) server_negotiation.cc:407] Connection header received
0811 00:12:57.243518 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:57.243523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:57.243594 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:57.243739 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:57.245112 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:57.245116 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:57.245119 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:12:57.245135 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:57.245171 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:12:57.245182 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:57.245340 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:57.245538 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 00:12:57.245585 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 00:12:57.247052 15882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:57.240810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:45027 (local address 127.7.94.2:57183)
0811 00:12:57.241739 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:57.241790 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:57.243255 (+ 1465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:57.243995 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:57.244003 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:57.244119 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:57.244792 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:12:57.244795 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:57.244837 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:12:57.244916 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:57.245328 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:57.245338 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:57.245345 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 00:12:57.245622 (+ 277us) client_negotiation.cc:240] Negotiation successful
0811 00:12:57.245890 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":273,"threads_started":1}
I20260811 00:12:57.247133 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45027
--builtin_ntp_servers=127.7.94.20:41249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 00:12:57.248404 15880 heartbeater.cc:347] Connected to a master server at 127.7.94.62:45027
I20260811 00:12:57.248695 15880 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:57.249331 15880 heartbeater.cc:511] Master 127.7.94.62:45027 requested a full tablet report, sending...
I20260811 00:12:57.250687 15357 ts_manager.cc:194] Registered new tserver with Master: 7313d798516c4c3b980b4d9fa2c492d3 (127.7.94.2:33153)
W20260811 00:12:57.576690 15884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:57.576944 15884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:57.577028 15884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:57.584062 15884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:57.584146 15884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:57.584328 15884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:12:57.593034 15884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:57.594238 15884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:57.595918 15884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:57.608955 15892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:57.609025 15897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:57.609035 15901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:57.610231 15884 server_base.cc:1034] running on GCE node
I20260811 00:12:57.610841 15884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:57.611721 15884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:57.612912 15884 hybrid_clock.cc:648] HybridClock initialized: now 1786407177612858 us; error 51 us; skew 500 ppm
I20260811 00:12:57.613245 15884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:12:57.615353 15884 webserver.cc:460] Webserver started at http://127.7.94.3:45979/ using document root <none> and password file <none>
I20260811 00:12:57.616004 15884 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:57.616119 15884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:57.616448 15884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:57.619213 15884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "c560d5defcf2409f8373a45ee4afe0f3"
format_stamp: "Formatted at 2026-08-11 00:12:57 on dist-test-slave-8hhm"
server_key: "f8705fe1111a869cea8ec608ec9663ea"
server_key_iv: "e52bf24297941e197b55e158e416d5b2"
server_key_version: "encryptionkey@0"
I20260811 00:12:57.619984 15884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "c560d5defcf2409f8373a45ee4afe0f3"
format_stamp: "Formatted at 2026-08-11 00:12:57 on dist-test-slave-8hhm"
server_key: "f8705fe1111a869cea8ec608ec9663ea"
server_key_iv: "e52bf24297941e197b55e158e416d5b2"
server_key_version: "encryptionkey@0"
I20260811 00:12:57.624740 15884 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260811 00:12:57.628016 15911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:57.629293 15884 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 00:12:57.629453 15884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "c560d5defcf2409f8373a45ee4afe0f3"
format_stamp: "Formatted at 2026-08-11 00:12:57 on dist-test-slave-8hhm"
server_key: "f8705fe1111a869cea8ec608ec9663ea"
server_key_iv: "e52bf24297941e197b55e158e416d5b2"
server_key_version: "encryptionkey@0"
I20260811 00:12:57.629690 15884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:57.635057 15884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:57.635794 15884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:57.636063 15884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:57.636938 15884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:12:57.638309 15884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:12:57.638401 15884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:57.638558 15884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:12:57.638662 15884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:57.669981 15884 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:46871
I20260811 00:12:57.670025 16099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:46871 every 8 connection(s)
I20260811 00:12:57.671345 15884 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:12:57.679217 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:57.674002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:58905 (local address 127.7.94.62:45027)
0811 00:12:57.674190 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:57.674194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:57.675514 (+ 1320us) server_negotiation.cc:407] Connection header received
0811 00:12:57.677153 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:57.677158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:57.677271 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:57.677357 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:57.678636 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:57.678640 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:57.678643 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:12:57.678662 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:57.678686 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:12:57.678697 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:57.678780 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:57.678945 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 00:12:57.679000 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 00:12:57.680069 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:57.674426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:45027 (local address 127.7.94.3:58905)
0811 00:12:57.675354 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:57.675401 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:57.676909 (+ 1508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:57.677557 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:57.677565 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:57.677656 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:57.678359 (+ 703us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:12:57.678362 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:57.678412 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:12:57.678494 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:57.678777 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:57.678784 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:57.678789 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 00:12:57.678989 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 00:12:57.679177 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":296,"threads_started":1}
I20260811 00:12:57.680757 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 15884
I20260811 00:12:57.680954 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:12:57.681542 7544 external_mini_cluster.cc:1428] Setting key d25a75cb3b30acb6c0a4ec22c6bc49c0
I20260811 00:12:57.681595 16101 heartbeater.cc:347] Connected to a master server at 127.7.94.62:45027
I20260811 00:12:57.681887 16101 heartbeater.cc:464] Registering TS with master...
I20260811 00:12:57.682499 16101 heartbeater.cc:511] Master 127.7.94.62:45027 requested a full tablet report, sending...
I20260811 00:12:57.683972 15356 ts_manager.cc:194] Registered new tserver with Master: c560d5defcf2409f8373a45ee4afe0f3 (127.7.94.3:46871)
I20260811 00:12:57.696518 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:12:57.703776 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:57.701100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59440 (local address 127.7.94.62:45027)
0811 00:12:57.701288 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:57.701292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:57.701816 (+ 524us) server_negotiation.cc:407] Connection header received
0811 00:12:57.702014 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:57.702019 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:57.702073 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:57.702180 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:57.703133 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:57.703138 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:57.703143 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:12:57.703161 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:57.703201 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:12:57.703212 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:57.703331 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:57.703447 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 00:12:57.703508 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 00:12:57.707428 15356 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59440:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:12:57.714241 15356 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:12:57.745328 16130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:57.727811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38799 (local address 127.0.0.1:54452)
0811 00:12:57.741709 (+ 13898us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:57.741733 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:57.741844 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:57.742955 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:57.742960 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:57.742985 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:57.743751 (+ 766us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:12:57.743755 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:57.743771 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:12:57.743816 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:57.744568 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:57.744573 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:57.744578 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 00:12:57.744732 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 00:12:57.744786 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13733,"thread_start_us":13570,"threads_started":1}
I20260811 00:12:57.745548 16131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:57.728090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54452 (local address 127.7.94.1:38799)
0811 00:12:57.742168 (+ 14078us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:57.742182 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:57.742225 (+ 43us) server_negotiation.cc:407] Connection header received
0811 00:12:57.742314 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:57.742326 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:57.742588 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:57.742880 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:57.744031 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:57.744043 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:57.744062 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:12:57.744084 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:57.744152 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:12:57.744179 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:57.744954 (+ 775us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:57.745052 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 00:12:57.745162 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13855,"thread_start_us":13674,"threads_started":1}
I20260811 00:12:57.745750 16129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:57.725899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57042 (local address 127.7.94.3:46871)
0811 00:12:57.739366 (+ 13467us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:57.739388 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:57.739424 (+ 36us) server_negotiation.cc:407] Connection header received
0811 00:12:57.739524 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:57.739534 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:57.739801 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:57.739960 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:57.744076 (+ 4116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:57.744090 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:57.744106 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:12:57.744129 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:57.744200 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:12:57.744226 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:57.745161 (+ 935us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:57.745285 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 00:12:57.745391 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13273,"thread_start_us":13892,"threads_started":1}
I20260811 00:12:57.746125 16124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:57.725568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:46871 (local address 127.0.0.1:57042)
0811 00:12:57.738571 (+ 13003us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:57.738596 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:57.738744 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:57.742128 (+ 3384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:57.742135 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:57.742219 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:57.743405 (+ 1186us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:12:57.743411 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:57.743447 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:12:57.743547 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:57.744463 (+ 916us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:57.744477 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:57.744485 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 00:12:57.744673 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 00:12:57.744777 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12808,"thread_start_us":16343,"threads_started":1}
I20260811 00:12:57.746799 16122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:57.725330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:33153 (local address 127.0.0.1:34014)
0811 00:12:57.726224 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:57.726283 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:57.726445 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:57.743931 (+ 17486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:57.743935 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:57.743957 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:57.744596 (+ 639us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:12:57.744598 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:57.744614 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:12:57.744661 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:57.746196 (+ 1535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:57.746201 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:57.746206 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 00:12:57.746328 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 00:12:57.746374 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":742,"threads_started":1}
I20260811 00:12:57.747673 16128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:57.725699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34014 (local address 127.7.94.2:33153)
0811 00:12:57.743171 (+ 17472us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:57.743183 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:57.743218 (+ 35us) server_negotiation.cc:407] Connection header received
0811 00:12:57.743294 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:57.743303 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:57.743545 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:57.744362 (+ 817us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:57.745958 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:57.745966 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:57.745977 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:12:57.745999 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:57.746045 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:12:57.746063 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:57.747126 (+ 1063us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:57.747203 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 00:12:57.747318 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17278,"thread_start_us":17143,"threads_started":1}
I20260811 00:12:57.749485 15556 tablet_service.cc:1467] Processing CreateTablet for tablet 217df3af3c87422da5e1da9130f755f6 (DEFAULT_TABLE table=test-table [id=4fa614a02f7441fdb23c629511637bc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:57.749915 15992 tablet_service.cc:1467] Processing CreateTablet for tablet 217df3af3c87422da5e1da9130f755f6 (DEFAULT_TABLE table=test-table [id=4fa614a02f7441fdb23c629511637bc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:57.751070 15556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 217df3af3c87422da5e1da9130f755f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:57.751245 15992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 217df3af3c87422da5e1da9130f755f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:57.751775 15777 tablet_service.cc:1467] Processing CreateTablet for tablet 217df3af3c87422da5e1da9130f755f6 (DEFAULT_TABLE table=test-table [id=4fa614a02f7441fdb23c629511637bc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:12:57.753113 15777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 217df3af3c87422da5e1da9130f755f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:57.762454 16132 tablet_bootstrap.cc:492] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3: Bootstrap starting.
I20260811 00:12:57.765748 16132 tablet_bootstrap.cc:654] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:57.767145 16135 tablet_bootstrap.cc:492] T 217df3af3c87422da5e1da9130f755f6 P ed957c7e078d47b89b841a55acbd7b70: Bootstrap starting.
I20260811 00:12:57.767163 16132 log.cc:824] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:57.768635 16136 tablet_bootstrap.cc:492] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3: Bootstrap starting.
I20260811 00:12:57.771124 16135 tablet_bootstrap.cc:654] T 217df3af3c87422da5e1da9130f755f6 P ed957c7e078d47b89b841a55acbd7b70: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:57.771823 16136 tablet_bootstrap.cc:654] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:57.772233 16132 tablet_bootstrap.cc:492] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3: No bootstrap required, opened a new log
I20260811 00:12:57.772440 16135 log.cc:824] T 217df3af3c87422da5e1da9130f755f6 P ed957c7e078d47b89b841a55acbd7b70: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:57.772531 16132 ts_tablet_manager.cc:1397] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 00:12:57.772879 16136 log.cc:824] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:57.775401 16135 tablet_bootstrap.cc:492] T 217df3af3c87422da5e1da9130f755f6 P ed957c7e078d47b89b841a55acbd7b70: No bootstrap required, opened a new log
I20260811 00:12:57.775681 16135 ts_tablet_manager.cc:1397] T 217df3af3c87422da5e1da9130f755f6 P ed957c7e078d47b89b841a55acbd7b70: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260811 00:12:57.775914 16136 tablet_bootstrap.cc:492] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3: No bootstrap required, opened a new log
I20260811 00:12:57.776217 16136 ts_tablet_manager.cc:1397] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260811 00:12:57.780462 16132 raft_consensus.cc:348] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7313d798516c4c3b980b4d9fa2c492d3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 33153 } } peers { permanent_uuid: "c560d5defcf2409f8373a45ee4afe0f3" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 46871 } } peers { permanent_uuid: "ed957c7e078d47b89b841a55acbd7b70" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38799 } }
I20260811 00:12:57.780835 16132 raft_consensus.cc:374] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:57.780959 16132 raft_consensus.cc:729] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c560d5defcf2409f8373a45ee4afe0f3, State: Initialized, Role: FOLLOWER
I20260811 00:12:57.781606 16132 consensus_queue.cc:260] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7313d798516c4c3b980b4d9fa2c492d3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 33153 } } peers { permanent_uuid: "c560d5defcf2409f8373a45ee4afe0f3" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 46871 } } peers { permanent_uuid: "ed957c7e078d47b89b841a55acbd7b70" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38799 } }
I20260811 00:12:57.782656 16101 heartbeater.cc:503] Master 127.7.94.62:45027 was elected leader, sending a full tablet report...
I20260811 00:12:57.783344 16132 ts_tablet_manager.cc:1428] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260811 00:12:57.786283 16135 raft_consensus.cc:348] T 217df3af3c87422da5e1da9130f755f6 P ed957c7e078d47b89b841a55acbd7b70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7313d798516c4c3b980b4d9fa2c492d3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 33153 } } peers { permanent_uuid: "c560d5defcf2409f8373a45ee4afe0f3" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 46871 } } peers { permanent_uuid: "ed957c7e078d47b89b841a55acbd7b70" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38799 } }
I20260811 00:12:57.786717 16135 raft_consensus.cc:374] T 217df3af3c87422da5e1da9130f755f6 P ed957c7e078d47b89b841a55acbd7b70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:57.786835 16135 raft_consensus.cc:729] T 217df3af3c87422da5e1da9130f755f6 P ed957c7e078d47b89b841a55acbd7b70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed957c7e078d47b89b841a55acbd7b70, State: Initialized, Role: FOLLOWER
I20260811 00:12:57.787107 16136 raft_consensus.cc:348] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7313d798516c4c3b980b4d9fa2c492d3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 33153 } } peers { permanent_uuid: "c560d5defcf2409f8373a45ee4afe0f3" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 46871 } } peers { permanent_uuid: "ed957c7e078d47b89b841a55acbd7b70" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38799 } }
I20260811 00:12:57.787571 16136 raft_consensus.cc:374] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:57.787699 16136 raft_consensus.cc:729] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7313d798516c4c3b980b4d9fa2c492d3, State: Initialized, Role: FOLLOWER
I20260811 00:12:57.787510 16135 consensus_queue.cc:260] T 217df3af3c87422da5e1da9130f755f6 P ed957c7e078d47b89b841a55acbd7b70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7313d798516c4c3b980b4d9fa2c492d3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 33153 } } peers { permanent_uuid: "c560d5defcf2409f8373a45ee4afe0f3" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 46871 } } peers { permanent_uuid: "ed957c7e078d47b89b841a55acbd7b70" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38799 } }
I20260811 00:12:57.788650 15659 heartbeater.cc:503] Master 127.7.94.62:45027 was elected leader, sending a full tablet report...
I20260811 00:12:57.788453 16136 consensus_queue.cc:260] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7313d798516c4c3b980b4d9fa2c492d3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 33153 } } peers { permanent_uuid: "c560d5defcf2409f8373a45ee4afe0f3" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 46871 } } peers { permanent_uuid: "ed957c7e078d47b89b841a55acbd7b70" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38799 } }
I20260811 00:12:57.789881 15880 heartbeater.cc:503] Master 127.7.94.62:45027 was elected leader, sending a full tablet report...
I20260811 00:12:57.790421 16136 ts_tablet_manager.cc:1428] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260811 00:12:57.790541 16135 ts_tablet_manager.cc:1428] T 217df3af3c87422da5e1da9130f755f6 P ed957c7e078d47b89b841a55acbd7b70: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
W20260811 00:12:57.804402 15661 tablet.cc:2369] T 217df3af3c87422da5e1da9130f755f6 P ed957c7e078d47b89b841a55acbd7b70: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:12:57.906229 16142 raft_consensus.cc:482] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:12:57.906642 16142 raft_consensus.cc:504] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7313d798516c4c3b980b4d9fa2c492d3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 33153 } } peers { permanent_uuid: "c560d5defcf2409f8373a45ee4afe0f3" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 46871 } } peers { permanent_uuid: "ed957c7e078d47b89b841a55acbd7b70" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38799 } }
I20260811 00:12:57.908531 16142 leader_election.cc:288] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7313d798516c4c3b980b4d9fa2c492d3 (127.7.94.2:33153), ed957c7e078d47b89b841a55acbd7b70 (127.7.94.1:38799)
I20260811 00:12:57.911545 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:57.908885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:33153 (local address 127.7.94.3:53651)
0811 00:12:57.909213 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:57.909252 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:57.909370 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:57.909808 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:57.909812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:57.909835 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:57.910591 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:12:57.910596 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:57.910623 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:12:57.910686 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:57.911086 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:57.911091 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:57.911102 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 00:12:57.911244 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 00:12:57.911297 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201}
I20260811 00:12:57.911556 16128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:57.908868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:53651 (local address 127.7.94.2:33153)
0811 00:12:57.909066 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:57.909072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:57.909354 (+ 282us) server_negotiation.cc:407] Connection header received
0811 00:12:57.909525 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:57.909535 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:57.909621 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:57.909732 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:57.910853 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:57.910857 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:57.910860 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:12:57.910898 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:57.910939 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:12:57.910956 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:57.911084 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:57.911243 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 00:12:57.911292 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 00:12:57.912235 16131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:57.909140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:36261 (local address 127.7.94.1:38799)
0811 00:12:57.909360 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:57.909365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:57.909993 (+ 628us) server_negotiation.cc:407] Connection header received
0811 00:12:57.910194 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:57.910202 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:57.910264 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:57.910370 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:57.911453 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:57.911459 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:57.911463 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:12:57.911480 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:57.911514 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:12:57.911528 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:57.911658 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:57.911896 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 00:12:57.911965 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 00:12:57.912331 16146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:57.909015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38799 (local address 127.7.94.3:36261)
0811 00:12:57.909883 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:57.909905 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:57.910035 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:57.910485 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:57.910493 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:57.910529 (+ 36us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:12:57.911237 (+ 708us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:12:57.911240 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:12:57.911258 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:12:57.911301 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:12:57.911662 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:12:57.911667 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:12:57.911672 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 00:12:57.911877 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 00:12:57.911921 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"thread_start_us":555,"threads_started":1}
I20260811 00:12:57.912765 15808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "217df3af3c87422da5e1da9130f755f6" candidate_uuid: "c560d5defcf2409f8373a45ee4afe0f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7313d798516c4c3b980b4d9fa2c492d3" is_pre_election: true
I20260811 00:12:57.913373 15808 raft_consensus.cc:2454] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c560d5defcf2409f8373a45ee4afe0f3 in term 0.
I20260811 00:12:57.913483 15586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "217df3af3c87422da5e1da9130f755f6" candidate_uuid: "c560d5defcf2409f8373a45ee4afe0f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed957c7e078d47b89b841a55acbd7b70" is_pre_election: true
I20260811 00:12:57.914070 15586 raft_consensus.cc:2454] T 217df3af3c87422da5e1da9130f755f6 P ed957c7e078d47b89b841a55acbd7b70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c560d5defcf2409f8373a45ee4afe0f3 in term 0.
I20260811 00:12:57.914151 15918 leader_election.cc:302] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7313d798516c4c3b980b4d9fa2c492d3, c560d5defcf2409f8373a45ee4afe0f3; no voters:
I20260811 00:12:57.914549 16142 raft_consensus.cc:2781] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:12:57.914732 16142 raft_consensus.cc:482] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:12:57.914845 16142 raft_consensus.cc:3037] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:57.917416 16142 raft_consensus.cc:504] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7313d798516c4c3b980b4d9fa2c492d3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 33153 } } peers { permanent_uuid: "c560d5defcf2409f8373a45ee4afe0f3" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 46871 } } peers { permanent_uuid: "ed957c7e078d47b89b841a55acbd7b70" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38799 } }
I20260811 00:12:57.918004 16142 leader_election.cc:288] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [CANDIDATE]: Term 1 election: Requested vote from peers 7313d798516c4c3b980b4d9fa2c492d3 (127.7.94.2:33153), ed957c7e078d47b89b841a55acbd7b70 (127.7.94.1:38799)
I20260811 00:12:57.918344 15808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "217df3af3c87422da5e1da9130f755f6" candidate_uuid: "c560d5defcf2409f8373a45ee4afe0f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7313d798516c4c3b980b4d9fa2c492d3"
I20260811 00:12:57.918548 15808 raft_consensus.cc:3037] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:57.918534 15586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "217df3af3c87422da5e1da9130f755f6" candidate_uuid: "c560d5defcf2409f8373a45ee4afe0f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed957c7e078d47b89b841a55acbd7b70"
I20260811 00:12:57.918761 15586 raft_consensus.cc:3037] T 217df3af3c87422da5e1da9130f755f6 P ed957c7e078d47b89b841a55acbd7b70 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:57.920773 15808 raft_consensus.cc:2454] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c560d5defcf2409f8373a45ee4afe0f3 in term 1.
I20260811 00:12:57.920920 15586 raft_consensus.cc:2454] T 217df3af3c87422da5e1da9130f755f6 P ed957c7e078d47b89b841a55acbd7b70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c560d5defcf2409f8373a45ee4afe0f3 in term 1.
I20260811 00:12:57.921169 15918 leader_election.cc:302] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7313d798516c4c3b980b4d9fa2c492d3, c560d5defcf2409f8373a45ee4afe0f3; no voters:
I20260811 00:12:57.921463 16142 raft_consensus.cc:2781] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:57.921900 16142 raft_consensus.cc:686] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [term 1 LEADER]: Becoming Leader. State: Replica: c560d5defcf2409f8373a45ee4afe0f3, State: Running, Role: LEADER
I20260811 00:12:57.922433 16142 consensus_queue.cc:237] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7313d798516c4c3b980b4d9fa2c492d3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 33153 } } peers { permanent_uuid: "c560d5defcf2409f8373a45ee4afe0f3" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 46871 } } peers { permanent_uuid: "ed957c7e078d47b89b841a55acbd7b70" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38799 } }
W20260811 00:12:57.922937 16103 tablet.cc:2369] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:12:57.927165 15357 catalog_manager.cc:5495] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to c560d5defcf2409f8373a45ee4afe0f3 (127.7.94.3). New cstate: current_term: 1 leader_uuid: "c560d5defcf2409f8373a45ee4afe0f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7313d798516c4c3b980b4d9fa2c492d3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 33153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c560d5defcf2409f8373a45ee4afe0f3" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 46871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed957c7e078d47b89b841a55acbd7b70" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38799 } health_report { overall_health: UNKNOWN } } }
I20260811 00:12:57.970309 16129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:57.968412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57052 (local address 127.7.94.3:46871)
0811 00:12:57.968591 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:57.968597 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:57.968612 (+ 15us) server_negotiation.cc:407] Connection header received
0811 00:12:57.968798 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:57.968801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:57.968877 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:57.968955 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:57.969861 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:57.969865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:57.969868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:12:57.969881 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:57.969910 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:12:57.969921 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:57.970027 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:57.970062 (+ 35us) server_negotiation.cc:299] Negotiation successful
0811 00:12:57.970103 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 00:12:57.978677 15808 raft_consensus.cc:1261] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3 [term 1 FOLLOWER]: Refusing update from remote peer c560d5defcf2409f8373a45ee4afe0f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:12:57.978679 15586 raft_consensus.cc:1261] T 217df3af3c87422da5e1da9130f755f6 P ed957c7e078d47b89b841a55acbd7b70 [term 1 FOLLOWER]: Refusing update from remote peer c560d5defcf2409f8373a45ee4afe0f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:12:57.979660 16147 consensus_queue.cc:1035] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7313d798516c4c3b980b4d9fa2c492d3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 33153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:12:57.980005 16142 consensus_queue.cc:1035] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed957c7e078d47b89b841a55acbd7b70" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 00:12:57.990018 15881 tablet.cc:2369] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:12:58.005362 15357 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:12:58.005663 15357 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:12:58.009234 15357 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ab7f0241ba1f4a55a197412ca12d690b: Sending DeleteTablet for 3 replicas of tablet 217df3af3c87422da5e1da9130f755f6
I20260811 00:12:58.010360 15777 tablet_service.cc:1514] Processing DeleteTablet for tablet 217df3af3c87422da5e1da9130f755f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:58 UTC) from {username='slave'} at 127.0.0.1:34014
I20260811 00:12:58.010372 15992 tablet_service.cc:1514] Processing DeleteTablet for tablet 217df3af3c87422da5e1da9130f755f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:58 UTC) from {username='slave'} at 127.0.0.1:57042
I20260811 00:12:58.010612 15556 tablet_service.cc:1514] Processing DeleteTablet for tablet 217df3af3c87422da5e1da9130f755f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:12:58 UTC) from {username='slave'} at 127.0.0.1:54452
I20260811 00:12:58.011524 16171 tablet_replica.cc:330] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3: stopping tablet replica
I20260811 00:12:58.011883 16171 raft_consensus.cc:2229] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:12:58.012409 16171 raft_consensus.cc:2258] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:58.012444 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 15461
I20260811 00:12:58.018263 16171 ts_tablet_manager.cc:1905] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:58.019345 16131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:58.013528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54458 (local address 127.7.94.1:38799)
0811 00:12:58.013702 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:58.013710 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:58.014900 (+ 1190us) server_negotiation.cc:407] Connection header received
0811 00:12:58.015105 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:58.015111 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:58.015214 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:58.015311 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:58.018502 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:58.018510 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:58.018515 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:12:58.018537 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:58.018576 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:12:58.018594 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:58.018697 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:58.018947 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 00:12:58.019009 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 00:12:58.020511 15653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:58.025197 16168 tablet_replica.cc:330] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3: stopping tablet replica
I20260811 00:12:58.029481 16168 raft_consensus.cc:2229] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:12:58.029978 16171 ts_tablet_manager.cc:1918] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:12:58.030175 16171 log.cc:1196] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/217df3af3c87422da5e1da9130f755f6
I20260811 00:12:58.030474 16168 raft_consensus.cc:2258] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:12:58.030815 16171 ts_tablet_manager.cc:1939] T 217df3af3c87422da5e1da9130f755f6 P 7313d798516c4c3b980b4d9fa2c492d3: Deleting consensus metadata
I20260811 00:12:58.032668 15341 catalog_manager.cc:4841] TS 7313d798516c4c3b980b4d9fa2c492d3 (127.7.94.2:33153): tablet 217df3af3c87422da5e1da9130f755f6 (table test-table [id=4fa614a02f7441fdb23c629511637bc1]) successfully deleted
I20260811 00:12:58.033134 16168 ts_tablet_manager.cc:1905] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:12:58.038563 16168 ts_tablet_manager.cc:1918] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:12:58.038751 16168 log.cc:1196] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/217df3af3c87422da5e1da9130f755f6
I20260811 00:12:58.039201 16168 ts_tablet_manager.cc:1939] T 217df3af3c87422da5e1da9130f755f6 P c560d5defcf2409f8373a45ee4afe0f3: Deleting consensus metadata
I20260811 00:12:58.040585 15340 catalog_manager.cc:4841] TS c560d5defcf2409f8373a45ee4afe0f3 (127.7.94.3:46871): tablet 217df3af3c87422da5e1da9130f755f6 (table test-table [id=4fa614a02f7441fdb23c629511637bc1]) successfully deleted
I20260811 00:12:58.133240 16172 tablet_replica.cc:330] T 217df3af3c87422da5e1da9130f755f6 P ed957c7e078d47b89b841a55acbd7b70: stopping tablet replica
I20260811 00:12:58.133459 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 15461
W20260811 00:12:58.169346 15336 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.94.1:38799 (error 108)
I20260811 00:12:58.170136 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 15665
I20260811 00:12:58.171742 16122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:58.171080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38799 (local address 127.0.0.1:54464)
0811 00:12:58.171413 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:58.171552 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:38799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197}
W20260811 00:12:58.171999 15336 catalog_manager.cc:4584] TS ed957c7e078d47b89b841a55acbd7b70 (127.7.94.1:38799): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 217df3af3c87422da5e1da9130f755f6: Network error: Client connection negotiation failed: client connection to 127.7.94.1:38799: connect: Connection refused (error 111)
I20260811 00:12:58.173367 16128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:58.171025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34024 (local address 127.7.94.2:33153)
0811 00:12:58.171290 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:58.171297 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:58.171320 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:12:58.171462 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:58.171469 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:58.171581 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:58.171717 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:58.172730 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:58.172734 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:58.172737 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:12:58.172752 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:58.172781 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:12:58.172792 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:58.172892 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:58.173015 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 00:12:58.173061 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260811 00:12:58.174093 15871 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:58.225739 16122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:58.225135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38799 (local address 127.0.0.1:54468)
0811 00:12:58.225391 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:58.225513 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:38799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260811 00:12:58.226897 16122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:58.226507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38799 (local address 127.0.0.1:54482)
0811 00:12:58.226676 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:58.226759 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:38799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 00:12:58.262187 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 15665
I20260811 00:12:58.287057 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 15884
I20260811 00:12:58.287621 16122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:58.287127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38799 (local address 127.0.0.1:54496)
0811 00:12:58.287362 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:58.287483 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:38799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 00:12:58.288795 16122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:58.288469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38799 (local address 127.0.0.1:54512)
0811 00:12:58.288617 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:58.288694 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:38799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 00:12:58.290480 16129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:58.287869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57066 (local address 127.7.94.3:46871)
0811 00:12:58.288031 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:58.288037 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:58.288170 (+ 133us) server_negotiation.cc:407] Connection header received
0811 00:12:58.288411 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:58.288417 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:58.288497 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:58.288592 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:58.289790 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:58.289795 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:58.289798 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:12:58.289812 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:58.289841 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:12:58.289851 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:58.289954 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:58.290127 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 00:12:58.290172 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 00:12:58.291194 16092 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:58.369809 16122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:58.369144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38799 (local address 127.0.0.1:54514)
0811 00:12:58.369486 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:58.369618 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:38799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197}
I20260811 00:12:58.370903 16122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:58.370532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38799 (local address 127.0.0.1:54516)
0811 00:12:58.370708 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:58.370799 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:38799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260811 00:12:58.410110 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 15884
I20260811 00:12:58.441988 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 15311
I20260811 00:12:58.443267 15410 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:58.548456 16122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:58.547778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38799 (local address 127.0.0.1:54520)
0811 00:12:58.548012 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:58.548137 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:38799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 00:12:58.548659 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 15311
2026-08-11T00:12:58Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2786 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 00:12:58.601450 7544 mini_oidc.cc:152] Starting JWKS server
I20260811 00:12:58.604254 7544 webserver.cc:460] Webserver started at https://127.0.0.1:45469/ using document root <none> and password file <none>
I20260811 00:12:58.604619 7544 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 00:12:58.605316 7544 webserver.cc:460] Webserver started at http://127.0.0.1:32837/ using document root <none> and password file <none>
2026-08-11T00:12:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:12:58Z Disabled control of system clock
I20260811 00:12:58.638676 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:42509
--webserver_interface=127.7.94.62
--webserver_port=0
--builtin_ntp_servers=127.7.94.20:42493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:42509
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45469/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407133955383-7544-0/testchainca.cert with env {}
W20260811 00:12:58.970131 16198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:58.970472 16198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:58.970642 16198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:58.977691 16198 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:12:58.977834 16198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:58.977928 16198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:58.978039 16198 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:12:58.978127 16198 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 00:12:58.978215 16198 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 00:12:58.978286 16198 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45469/jwks.json
I20260811 00:12:58.987947 16198 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:42493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:42509
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:42509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45469/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407133955383-7544-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:58.989310 16198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:58.990620 16198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:59.002482 16208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:59.002509 16205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:59.002561 16204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:59.003528 16198 server_base.cc:1034] running on GCE node
I20260811 00:12:59.004354 16198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:59.005832 16198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:59.007062 16198 hybrid_clock.cc:648] HybridClock initialized: now 1786407179007005 us; error 48 us; skew 500 ppm
I20260811 00:12:59.007440 16198 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:12:59.014485 16198 webserver.cc:460] Webserver started at http://127.7.94.62:36739/ using document root <none> and password file <none>
I20260811 00:12:59.015229 16198 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:59.015359 16198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:59.015698 16198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:59.018677 16198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "bc7b6172fa9d4cfca9e767f240453336"
format_stamp: "Formatted at 2026-08-11 00:12:59 on dist-test-slave-8hhm"
server_key: "1246490e8b820eff44d30c21f0d37f83"
server_key_iv: "6b57962252861a11b43cade3b6e642d5"
server_key_version: "encryptionkey@0"
I20260811 00:12:59.019541 16198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "bc7b6172fa9d4cfca9e767f240453336"
format_stamp: "Formatted at 2026-08-11 00:12:59 on dist-test-slave-8hhm"
server_key: "1246490e8b820eff44d30c21f0d37f83"
server_key_iv: "6b57962252861a11b43cade3b6e642d5"
server_key_version: "encryptionkey@0"
I20260811 00:12:59.024595 16198 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 00:12:59.027987 16215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:59.029475 16198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 00:12:59.029661 16198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "bc7b6172fa9d4cfca9e767f240453336"
format_stamp: "Formatted at 2026-08-11 00:12:59 on dist-test-slave-8hhm"
server_key: "1246490e8b820eff44d30c21f0d37f83"
server_key_iv: "6b57962252861a11b43cade3b6e642d5"
server_key_version: "encryptionkey@0"
I20260811 00:12:59.029881 16198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:59.043342 16198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:59.044116 16198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:59.044402 16198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:59.062598 16198 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:42509
I20260811 00:12:59.062616 16290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:42509 every 8 connection(s)
I20260811 00:12:59.063906 16198 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:12:59.067608 16293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:59.069366 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 16198
I20260811 00:12:59.069590 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:12:59.070369 7544 external_mini_cluster.cc:1428] Setting key 386c6324a1a824d56ef9260bdaf955a9
I20260811 00:12:59.080452 16293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336: Bootstrap starting.
I20260811 00:12:59.083441 16301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:59.072355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54582 (local address 127.7.94.62:42509)
0811 00:12:59.073375 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:59.073386 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:59.073426 (+ 40us) server_negotiation.cc:407] Connection header received
0811 00:12:59.075031 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:59.075056 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:59.075494 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:59.075873 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:59.076499 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:59.077818 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:59.079125 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:59.079767 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:59.079971 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:59.081258 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:59.081269 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:59.081287 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:12:59.081823 (+ 536us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:59.081896 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:12:59.081933 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:59.082067 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:59.082300 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 00:12:59.082587 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":326,"threads_started":1}
I20260811 00:12:59.083639 16293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:59.084743 16293 log.cc:824] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:59.087347 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:12:59.087812 16293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336: No bootstrap required, opened a new log
I20260811 00:12:59.095302 16293 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc7b6172fa9d4cfca9e767f240453336" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42509 } }
I20260811 00:12:59.095638 16293 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:59.095762 16293 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc7b6172fa9d4cfca9e767f240453336, State: Initialized, Role: FOLLOWER
I20260811 00:12:59.096376 16293 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc7b6172fa9d4cfca9e767f240453336" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42509 } }
I20260811 00:12:59.096650 16293 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:12:59.096800 16293 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:12:59.096979 16293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:59.099187 16293 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc7b6172fa9d4cfca9e767f240453336" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42509 } }
I20260811 00:12:59.099653 16293 leader_election.cc:302] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc7b6172fa9d4cfca9e767f240453336; no voters:
I20260811 00:12:59.100215 16293 leader_election.cc:288] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:12:59.100351 16304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:59.101028 16304 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336 [term 1 LEADER]: Becoming Leader. State: Replica: bc7b6172fa9d4cfca9e767f240453336, State: Running, Role: LEADER
I20260811 00:12:59.101513 16304 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc7b6172fa9d4cfca9e767f240453336" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42509 } }
I20260811 00:12:59.102270 16293 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:12:59.104290 16307 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc7b6172fa9d4cfca9e767f240453336. Latest consensus state: current_term: 1 leader_uuid: "bc7b6172fa9d4cfca9e767f240453336" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc7b6172fa9d4cfca9e767f240453336" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42509 } } }
I20260811 00:12:59.105024 16307 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336 [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:59.104583 16306 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc7b6172fa9d4cfca9e767f240453336" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc7b6172fa9d4cfca9e767f240453336" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42509 } } }
I20260811 00:12:59.105388 16306 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336 [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:59.107622 16310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:12:59.113176 16310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:12:59.122093 16310 catalog_manager.cc:1347] Generated new cluster ID: ebb94462f3954215aa6ba46adc122579
I20260811 00:12:59.122210 16310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:12:59.136189 16310 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:12:59.138024 16310 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:12:59.150449 16310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc7b6172fa9d4cfca9e767f240453336: Generated new TSK 0
I20260811 00:12:59.151469 16310 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 00:12:59.204546 16343 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 00:12:59.204783 16343 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 00:12:59.206952 16301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:59.183666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54596 (local address 127.7.94.62:42509)
0811 00:12:59.183881 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:59.183888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:59.184346 (+ 458us) server_negotiation.cc:407] Connection header received
0811 00:12:59.184699 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:59.184706 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:59.184810 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:59.184971 (+ 161us) server_negotiation.cc:226] Negotiated authn=JWT
0811 00:12:59.185533 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:59.186733 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:59.188174 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:59.188873 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:59.189005 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:59.189594 (+ 589us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 00:12:59.206244 (+ 16650us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45469/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 00:12:59.206647 (+ 403us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54596: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45469/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 00:12:59.207084 16301 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54596: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45469/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 00:12:59.208719 16341 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:12:59.183504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:42509 (local address 127.0.0.1:54596)
0811 00:12:59.184241 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:59.184258 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:59.184410 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:59.184999 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:59.185002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:59.185012 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0811 00:12:59.185380 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:59.185391 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:59.186882 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:59.186886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:59.188025 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:59.188036 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:59.188994 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:59.189049 (+ 55us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:59.189172 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:59.189288 (+ 116us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 00:12:59.206559 (+ 17271us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45469/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 00:12:59.206835 (+ 276us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.94.62:42509: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45469/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":401,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.94.62:42509: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45469/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 00:12:59.210422 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 16198
I20260811 00:12:59.211598 16283 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:12:59.307155 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 16198
2026-08-11T00:12:59Z chronyd exiting
I20260811 00:12:59.364647 7544 test_util.cc:175] -----------------------------------------------
I20260811 00:12:59.364828 7544 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786407133955383-7544-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (768 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 00:12:59.368741 7544 mini_oidc.cc:152] Starting JWKS server
I20260811 00:12:59.370635 7544 webserver.cc:460] Webserver started at https://127.0.0.1:37853/ using document root <none> and password file <none>
I20260811 00:12:59.370879 7544 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 00:12:59.371563 7544 webserver.cc:460] Webserver started at http://127.0.0.1:45803/ using document root <none> and password file <none>
2026-08-11T00:12:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:12:59Z Disabled control of system clock
I20260811 00:12:59.404192 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:44053
--webserver_interface=127.7.94.62
--webserver_port=0
--builtin_ntp_servers=127.7.94.20:45301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:44053
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37853/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407133955383-7544-0/testchainca.cert with env {}
W20260811 00:12:59.734530 16354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:12:59.734889 16354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:12:59.735074 16354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:12:59.743981 16354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:12:59.744092 16354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:12:59.744133 16354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:12:59.744225 16354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:12:59.744325 16354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 00:12:59.744433 16354 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 00:12:59.744509 16354 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37853/jwks.json
I20260811 00:12:59.754793 16354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:45301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:44053
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:44053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37853/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407133955383-7544-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:12:59.756212 16354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:12:59.757637 16354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:12:59.769868 16370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:59.769929 16364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:12:59.770149 16363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:12:59.770819 16354 server_base.cc:1034] running on GCE node
I20260811 00:12:59.771615 16354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:12:59.772982 16354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:12:59.774215 16354 hybrid_clock.cc:648] HybridClock initialized: now 1786407179774179 us; error 27 us; skew 500 ppm
I20260811 00:12:59.774559 16354 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:12:59.776852 16354 webserver.cc:460] Webserver started at http://127.7.94.62:33901/ using document root <none> and password file <none>
I20260811 00:12:59.777618 16354 fs_manager.cc:359] Metadata directory not provided
I20260811 00:12:59.777758 16354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:12:59.778095 16354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:12:59.780833 16354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "a2b280b6a3194682897a6af9ef932962"
format_stamp: "Formatted at 2026-08-11 00:12:59 on dist-test-slave-8hhm"
server_key: "39f7625ce2623bd2851242738071f0da"
server_key_iv: "24bc97e21a4306e6ef1238cb7514dc77"
server_key_version: "encryptionkey@0"
I20260811 00:12:59.781661 16354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "a2b280b6a3194682897a6af9ef932962"
format_stamp: "Formatted at 2026-08-11 00:12:59 on dist-test-slave-8hhm"
server_key: "39f7625ce2623bd2851242738071f0da"
server_key_iv: "24bc97e21a4306e6ef1238cb7514dc77"
server_key_version: "encryptionkey@0"
I20260811 00:12:59.786996 16354 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 00:12:59.790319 16384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:12:59.791662 16354 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:12:59.791862 16354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "a2b280b6a3194682897a6af9ef932962"
format_stamp: "Formatted at 2026-08-11 00:12:59 on dist-test-slave-8hhm"
server_key: "39f7625ce2623bd2851242738071f0da"
server_key_iv: "24bc97e21a4306e6ef1238cb7514dc77"
server_key_version: "encryptionkey@0"
I20260811 00:12:59.792089 16354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:12:59.806185 16354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:12:59.806953 16354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:12:59.807252 16354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:12:59.825850 16354 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:44053
I20260811 00:12:59.825856 16464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:44053 every 8 connection(s)
I20260811 00:12:59.827229 16354 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:12:59.831013 16465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:12:59.834514 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 16354
I20260811 00:12:59.834733 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:12:59.835381 7544 external_mini_cluster.cc:1428] Setting key 13dd4876c84811f8af386859aa5bdaf0
I20260811 00:12:59.844774 16465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962: Bootstrap starting.
I20260811 00:12:59.848686 16465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962: Neither blocks nor log segments found. Creating new log.
I20260811 00:12:59.850126 16465 log.cc:824] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962: Log is configured to *not* fsync() on all Append() calls
I20260811 00:12:59.850528 16469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:59.837315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53154 (local address 127.7.94.62:44053)
0811 00:12:59.838468 (+ 1153us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:59.838484 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:59.838538 (+ 54us) server_negotiation.cc:407] Connection header received
0811 00:12:59.840319 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:59.840354 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:59.840970 (+ 616us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:59.841454 (+ 484us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:12:59.842033 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:59.843924 (+ 1891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:59.845346 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:59.846295 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:59.846527 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:59.847592 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:12:59.847602 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:12:59.847619 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:12:59.848350 (+ 731us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:12:59.848481 (+ 131us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:12:59.848518 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:12:59.848675 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 00:12:59.848853 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 00:12:59.849204 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":424,"threads_started":1}
I20260811 00:12:59.853181 16465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962: No bootstrap required, opened a new log
I20260811 00:12:59.854252 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:12:59.860986 16465 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2b280b6a3194682897a6af9ef932962" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44053 } }
I20260811 00:12:59.861416 16465 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:12:59.861514 16465 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b280b6a3194682897a6af9ef932962, State: Initialized, Role: FOLLOWER
I20260811 00:12:59.862119 16465 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2b280b6a3194682897a6af9ef932962" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44053 } }
I20260811 00:12:59.862409 16465 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:12:59.862555 16465 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:12:59.862732 16465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:12:59.864790 16465 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2b280b6a3194682897a6af9ef932962" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44053 } }
I20260811 00:12:59.865309 16465 leader_election.cc:302] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2b280b6a3194682897a6af9ef932962; no voters:
I20260811 00:12:59.865912 16465 leader_election.cc:288] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:12:59.866077 16471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:12:59.866806 16471 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962 [term 1 LEADER]: Becoming Leader. State: Replica: a2b280b6a3194682897a6af9ef932962, State: Running, Role: LEADER
I20260811 00:12:59.867292 16471 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2b280b6a3194682897a6af9ef932962" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44053 } }
I20260811 00:12:59.867934 16465 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:12:59.870510 16472 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2b280b6a3194682897a6af9ef932962" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2b280b6a3194682897a6af9ef932962" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44053 } } }
I20260811 00:12:59.870867 16472 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962 [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:59.874776 16473 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2b280b6a3194682897a6af9ef932962. Latest consensus state: current_term: 1 leader_uuid: "a2b280b6a3194682897a6af9ef932962" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2b280b6a3194682897a6af9ef932962" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44053 } } }
I20260811 00:12:59.875166 16473 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962 [sys.catalog]: This master's current role is: LEADER
I20260811 00:12:59.875597 16479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:12:59.879930 16479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:12:59.888893 16479 catalog_manager.cc:1347] Generated new cluster ID: 633a80494c904670823cf442111e82a4
I20260811 00:12:59.888991 16479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:12:59.912340 16479 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:12:59.913666 16479 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:12:59.924835 16479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2b280b6a3194682897a6af9ef932962: Generated new TSK 0
I20260811 00:12:59.925678 16479 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 00:12:59.956634 16505 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 00:12:59.956753 16505 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 00:12:59.957959 16469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:12:59.939397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53168 (local address 127.7.94.62:44053)
0811 00:12:59.939624 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 00:12:59.939631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:12:59.939947 (+ 316us) server_negotiation.cc:407] Connection header received
0811 00:12:59.940211 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:12:59.940218 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:12:59.940291 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:12:59.940435 (+ 144us) server_negotiation.cc:226] Negotiated authn=JWT
0811 00:12:59.940915 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:59.941697 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:59.943020 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:12:59.943464 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:12:59.943601 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:59.944069 (+ 468us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 00:12:59.957256 (+ 13187us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37853/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 00:12:59.957662 (+ 406us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53168: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37853/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":74}
W20260811 00:12:59.958087 16469 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53168: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37853/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 00:12:59.958163 16504 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:12:59.939269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:44053 (local address 127.0.0.1:53168)
0811 00:12:59.939847 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0811 00:12:59.939866 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:12:59.940028 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:12:59.940451 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:12:59.940456 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:12:59.940467 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0811 00:12:59.940746 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:59.940754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:59.941835 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:59.941840 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:59.942882 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:12:59.942891 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:12:59.943589 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:12:59.943633 (+ 44us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:12:59.943790 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:12:59.943820 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 00:12:59.957485 (+ 13665us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37853/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 00:12:59.957713 (+ 228us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.94.62:44053: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37853/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":409,"spinlock_wait_cycles":81280,"thread_start_us":390,"threads_started":1}
I20260811 00:12:59.959275 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 16354
I20260811 00:12:59.960410 16459 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:00.051649 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 16354
2026-08-11T00:13:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (745 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 00:13:00.113912 7544 mini_oidc.cc:152] Starting JWKS server
I20260811 00:13:00.115934 7544 webserver.cc:460] Webserver started at https://127.0.0.1:36291/ using document root <none> and password file <none>
I20260811 00:13:00.116225 7544 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 00:13:00.116904 7544 webserver.cc:460] Webserver started at http://127.0.0.1:34761/ using document root <none> and password file <none>
2026-08-11T00:13:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:13:00Z Disabled control of system clock
I20260811 00:13:00.151243 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:42803
--webserver_interface=127.7.94.62
--webserver_port=0
--builtin_ntp_servers=127.7.94.20:42363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:42803
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36291/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 00:13:00.468293 16517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:00.468721 16517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:00.468883 16517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:00.475596 16517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:13:00.475679 16517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:00.475713 16517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:00.475759 16517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:13:00.475790 16517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 00:13:00.475889 16517 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 00:13:00.475963 16517 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36291/jwks.json
I20260811 00:13:00.484431 16517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:42363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:42803
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:42803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36291/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:00.485656 16517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:00.487354 16517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:00.499508 16527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:00.499598 16528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:00.499626 16530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:00.500619 16517 server_base.cc:1034] running on GCE node
I20260811 00:13:00.501505 16517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:00.502890 16517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:00.504127 16517 hybrid_clock.cc:648] HybridClock initialized: now 1786407180504059 us; error 52 us; skew 500 ppm
I20260811 00:13:00.504521 16517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:13:00.506913 16517 webserver.cc:460] Webserver started at http://127.7.94.62:44929/ using document root <none> and password file <none>
I20260811 00:13:00.507642 16517 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:00.507783 16517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:00.508244 16517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:00.511291 16517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "fcc7329014fe44298a68f0f38b94b8c1"
format_stamp: "Formatted at 2026-08-11 00:13:00 on dist-test-slave-8hhm"
server_key: "19e991cfd2bdca933511d84d468898e4"
server_key_iv: "8cfc154fd49188759efda091e024b3e3"
server_key_version: "encryptionkey@0"
I20260811 00:13:00.512189 16517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "fcc7329014fe44298a68f0f38b94b8c1"
format_stamp: "Formatted at 2026-08-11 00:13:00 on dist-test-slave-8hhm"
server_key: "19e991cfd2bdca933511d84d468898e4"
server_key_iv: "8cfc154fd49188759efda091e024b3e3"
server_key_version: "encryptionkey@0"
I20260811 00:13:00.517477 16517 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260811 00:13:00.520951 16540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:00.522301 16517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:13:00.522511 16517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "fcc7329014fe44298a68f0f38b94b8c1"
format_stamp: "Formatted at 2026-08-11 00:13:00 on dist-test-slave-8hhm"
server_key: "19e991cfd2bdca933511d84d468898e4"
server_key_iv: "8cfc154fd49188759efda091e024b3e3"
server_key_version: "encryptionkey@0"
I20260811 00:13:00.522737 16517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:00.551657 16517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:00.552592 16517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:00.552942 16517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:00.572309 16517 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:42803
I20260811 00:13:00.572328 16627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:42803 every 8 connection(s)
I20260811 00:13:00.573720 16517 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:13:00.577471 16628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:00.580964 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 16517
I20260811 00:13:00.581200 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:13:00.582011 7544 external_mini_cluster.cc:1428] Setting key 33c3bbe5f897e0b91f3bf2676ca2b2ce
I20260811 00:13:00.590404 16628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1: Bootstrap starting.
I20260811 00:13:00.593950 16628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:00.595103 16628 log.cc:824] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1: Log is configured to *not* fsync() on all Append() calls
I20260811 00:13:00.596458 16631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:00.583994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51250 (local address 127.7.94.62:42803)
0811 00:13:00.585004 (+ 1010us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:00.585014 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:00.585055 (+ 41us) server_negotiation.cc:407] Connection header received
0811 00:13:00.586696 (+ 1641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:00.586717 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:00.587183 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:00.587547 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:00.588124 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:00.589689 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:00.591503 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:00.592231 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:00.592393 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:00.593656 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:00.593664 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:00.593678 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:13:00.594332 (+ 654us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:00.594429 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:13:00.594454 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:00.594550 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:00.594780 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 00:13:00.595141 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":389,"threads_started":1}
I20260811 00:13:00.598014 16628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1: No bootstrap required, opened a new log
I20260811 00:13:00.600395 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:13:00.606225 16628 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc7329014fe44298a68f0f38b94b8c1" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42803 } }
I20260811 00:13:00.606647 16628 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:00.606762 16628 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcc7329014fe44298a68f0f38b94b8c1, State: Initialized, Role: FOLLOWER
I20260811 00:13:00.607362 16628 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc7329014fe44298a68f0f38b94b8c1" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42803 } }
I20260811 00:13:00.607580 16628 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:13:00.607676 16628 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:13:00.607841 16628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:00.610044 16628 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc7329014fe44298a68f0f38b94b8c1" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42803 } }
I20260811 00:13:00.610491 16628 leader_election.cc:302] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcc7329014fe44298a68f0f38b94b8c1; no voters:
I20260811 00:13:00.610996 16628 leader_election.cc:288] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:13:00.611140 16636 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:13:00.611917 16636 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1 [term 1 LEADER]: Becoming Leader. State: Replica: fcc7329014fe44298a68f0f38b94b8c1, State: Running, Role: LEADER
I20260811 00:13:00.612459 16636 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc7329014fe44298a68f0f38b94b8c1" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42803 } }
I20260811 00:13:00.613080 16628 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:13:00.615298 16641 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcc7329014fe44298a68f0f38b94b8c1. Latest consensus state: current_term: 1 leader_uuid: "fcc7329014fe44298a68f0f38b94b8c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc7329014fe44298a68f0f38b94b8c1" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42803 } } }
I20260811 00:13:00.615311 16638 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcc7329014fe44298a68f0f38b94b8c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc7329014fe44298a68f0f38b94b8c1" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 42803 } } }
I20260811 00:13:00.615711 16641 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1 [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:00.615711 16638 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1 [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:00.619076 16644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:13:00.625262 16644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:13:00.636576 16644 catalog_manager.cc:1347] Generated new cluster ID: 6b1ff86cdd4441858bdac0bbeb8737a0
I20260811 00:13:00.636689 16644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:13:00.649888 16644 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:13:00.651722 16644 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:13:00.677203 16644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcc7329014fe44298a68f0f38b94b8c1: Generated new TSK 0
I20260811 00:13:00.678087 16644 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:13:00.706053 16631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:00.697744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51264 (local address 127.7.94.62:42803)
0811 00:13:00.697978 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:00.697984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:00.698378 (+ 394us) server_negotiation.cc:407] Connection header received
0811 00:13:00.698607 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:00.698611 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:00.698683 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:00.698809 (+ 126us) server_negotiation.cc:226] Negotiated authn=JWT
0811 00:13:00.699336 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:00.699994 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:00.702062 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:00.702503 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:00.702593 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:00.703196 (+ 603us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 00:13:00.705462 (+ 2266us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36291/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 00:13:00.705772 (+ 310us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51264: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36291/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 00:13:00.706171 16631 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51264: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36291/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 00:13:00.708189 16675 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:00.697623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:42803 (local address 127.0.0.1:51264)
0811 00:13:00.698251 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:00.698272 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:00.698436 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:00.698857 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:00.698862 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:00.698875 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0811 00:13:00.699188 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:00.699202 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:00.700130 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:00.700136 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:00.701919 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:00.701936 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:00.702622 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:00.702700 (+ 78us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:00.702899 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:00.702945 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 00:13:00.705693 (+ 2748us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36291/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 00:13:00.707751 (+ 2058us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.94.62:42803: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36291/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":1752,"thread_start_us":325,"threads_started":1}
I20260811 00:13:00.709517 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 16517
I20260811 00:13:00.710687 16620 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 00:13:00.720773 16678 webserver.cc:533] Webserver: sslize failed:
E20260811 00:13:00.720887 16678 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 00:13:00.803875 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 16517
2026-08-11T00:13:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (754 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T00:13:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:13:00Z Disabled control of system clock
I20260811 00:13:00.902763 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:36119
--webserver_interface=127.7.94.62
--webserver_port=0
--builtin_ntp_servers=127.7.94.20:40327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:36119
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407133955383-7544-0/testchainca.cert with env {}
W20260811 00:13:01.224716 16686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:01.225009 16686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:01.225095 16686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:01.232177 16686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:13:01.232275 16686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:01.232312 16686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:01.232374 16686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:13:01.232410 16686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 00:13:01.232476 16686 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 00:13:01.232515 16686 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 00:13:01.242383 16686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:40327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:36119
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:36119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407133955383-7544-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:01.243809 16686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:01.245000 16686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:01.256392 16692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:01.256505 16693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:01.256757 16697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:01.257491 16686 server_base.cc:1034] running on GCE node
I20260811 00:13:01.258320 16686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:01.259614 16686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:01.260850 16686 hybrid_clock.cc:648] HybridClock initialized: now 1786407181260796 us; error 49 us; skew 500 ppm
I20260811 00:13:01.261210 16686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:13:01.263547 16686 webserver.cc:460] Webserver started at http://127.7.94.62:43307/ using document root <none> and password file <none>
I20260811 00:13:01.264240 16686 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:01.264376 16686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:01.264695 16686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:01.267516 16686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "13226c6b46c944b2a2c2ef650cef6a48"
format_stamp: "Formatted at 2026-08-11 00:13:01 on dist-test-slave-8hhm"
server_key: "9d4df4ab54f702d9af7e7acf72e32678"
server_key_iv: "5d2f8e62447c81f154d7cb7bcec2e633"
server_key_version: "encryptionkey@0"
I20260811 00:13:01.268312 16686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "13226c6b46c944b2a2c2ef650cef6a48"
format_stamp: "Formatted at 2026-08-11 00:13:01 on dist-test-slave-8hhm"
server_key: "9d4df4ab54f702d9af7e7acf72e32678"
server_key_iv: "5d2f8e62447c81f154d7cb7bcec2e633"
server_key_version: "encryptionkey@0"
I20260811 00:13:01.273121 16686 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260811 00:13:01.276285 16704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:01.277652 16686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:13:01.277835 16686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "13226c6b46c944b2a2c2ef650cef6a48"
format_stamp: "Formatted at 2026-08-11 00:13:01 on dist-test-slave-8hhm"
server_key: "9d4df4ab54f702d9af7e7acf72e32678"
server_key_iv: "5d2f8e62447c81f154d7cb7bcec2e633"
server_key_version: "encryptionkey@0"
I20260811 00:13:01.278046 16686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:01.317024 16686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:01.317957 16686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:01.318260 16686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:01.336124 16686 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:36119
I20260811 00:13:01.336133 16813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:36119 every 8 connection(s)
I20260811 00:13:01.337471 16686 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:13:01.341002 16816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:01.341300 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 16686
I20260811 00:13:01.341495 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:13:01.342332 7544 external_mini_cluster.cc:1428] Setting key b767de817edd28f3855450e558c90c52
I20260811 00:13:01.354323 16816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48: Bootstrap starting.
I20260811 00:13:01.355731 16820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:01.344262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57744 (local address 127.7.94.62:36119)
0811 00:13:01.345332 (+ 1070us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:01.345349 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:01.345390 (+ 41us) server_negotiation.cc:407] Connection header received
0811 00:13:01.347152 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:01.347174 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:01.347630 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:01.347945 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:01.348502 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:01.349830 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:01.351629 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:01.352294 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:01.352413 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:01.353582 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:01.353589 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:01.353609 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:13:01.354090 (+ 481us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:01.354171 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:13:01.354191 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:01.354278 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:01.354552 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 00:13:01.354867 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":426,"threads_started":1}
I20260811 00:13:01.357429 16816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:01.358566 16816 log.cc:824] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48: Log is configured to *not* fsync() on all Append() calls
I20260811 00:13:01.359403 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:13:01.361680 16816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48: No bootstrap required, opened a new log
I20260811 00:13:01.368606 16816 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13226c6b46c944b2a2c2ef650cef6a48" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36119 } }
I20260811 00:13:01.368984 16816 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:01.369131 16816 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13226c6b46c944b2a2c2ef650cef6a48, State: Initialized, Role: FOLLOWER
I20260811 00:13:01.369695 16816 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13226c6b46c944b2a2c2ef650cef6a48" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36119 } }
I20260811 00:13:01.369925 16816 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:13:01.370067 16816 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:13:01.370235 16816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:01.372449 16816 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13226c6b46c944b2a2c2ef650cef6a48" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36119 } }
I20260811 00:13:01.372906 16816 leader_election.cc:302] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13226c6b46c944b2a2c2ef650cef6a48; no voters:
I20260811 00:13:01.373517 16816 leader_election.cc:288] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:13:01.373668 16822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:13:01.374346 16822 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48 [term 1 LEADER]: Becoming Leader. State: Replica: 13226c6b46c944b2a2c2ef650cef6a48, State: Running, Role: LEADER
I20260811 00:13:01.374837 16822 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13226c6b46c944b2a2c2ef650cef6a48" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36119 } }
I20260811 00:13:01.375483 16816 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:13:01.377668 16823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13226c6b46c944b2a2c2ef650cef6a48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13226c6b46c944b2a2c2ef650cef6a48" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36119 } } }
I20260811 00:13:01.378113 16823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48 [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:01.377959 16827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13226c6b46c944b2a2c2ef650cef6a48. Latest consensus state: current_term: 1 leader_uuid: "13226c6b46c944b2a2c2ef650cef6a48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13226c6b46c944b2a2c2ef650cef6a48" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36119 } } }
I20260811 00:13:01.378306 16827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48 [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:01.379669 16835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:13:01.385190 16835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:13:01.394008 16835 catalog_manager.cc:1347] Generated new cluster ID: 4f4174ebb4744bc685be73e3f6bb3245
I20260811 00:13:01.394124 16835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:13:01.414665 16835 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:13:01.416379 16835 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:13:01.426793 16835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13226c6b46c944b2a2c2ef650cef6a48: Generated new TSK 0
I20260811 00:13:01.427793 16835 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:13:01.506384 16820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:01.442722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57752 (local address 127.7.94.62:36119)
0811 00:13:01.442890 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:01.442898 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:01.443268 (+ 370us) server_negotiation.cc:407] Connection header received
0811 00:13:01.443447 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:01.443453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:01.443542 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:01.443713 (+ 171us) server_negotiation.cc:226] Negotiated authn=JWT
0811 00:13:01.444161 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:01.445381 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:01.446745 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:01.447508 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:01.447661 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:01.448069 (+ 408us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 00:13:01.505649 (+ 57580us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 00:13:01.506052 (+ 403us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57752: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 00:13:01.506536 16820 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57752: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 00:13:01.506556 16868 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:01.442582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36119 (local address 127.0.0.1:57752)
0811 00:13:01.443169 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:01.443185 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:01.443295 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:01.443735 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:01.443738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:01.443747 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 00:13:01.444022 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:01.444030 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:01.445522 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:01.445525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:01.446605 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:01.446615 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:01.447642 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:01.447685 (+ 43us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:01.447800 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:01.447830 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 00:13:01.505971 (+ 58141us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 00:13:01.506213 (+ 242us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.94.62:36119: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":314,"threads_started":1}
I20260811 00:13:01.507802 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 16686
I20260811 00:13:01.509028 16804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:01.606065 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 16686
2026-08-11T00:13:01Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (781 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (418 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (1575 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:13:03 dist-test-slave-8hhm krb5kdc[16898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:13:03 dist-test-slave-8hhm krb5kdc[16898](info): set up 2 sockets
Aug 11 00:13:03 dist-test-slave-8hhm krb5kdc[16898](info): commencing operation
krb5kdc: starting...
W20260811 00:13:05.078349 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.433s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:13:05 dist-test-slave-8hhm krb5kdc[16898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:13:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:13:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:13:05.330899 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:45677
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:46289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:45677
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:05.666059 16921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:05.666342 16921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:05.666523 16921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:05.673863 16921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:13:05.673985 16921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:05.674098 16921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:05.674188 16921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:13:05.674263 16921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:13:05.684343 16921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:46289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:45677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:45677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:05.685751 16921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:05.687145 16921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:05.699272 16932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:05.699278 16927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:05.699318 16930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:05.700428 16921 server_base.cc:1034] running on GCE node
I20260811 00:13:05.701206 16921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:05.702610 16921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:05.703850 16921 hybrid_clock.cc:648] HybridClock initialized: now 1786407185703817 us; error 43 us; skew 500 ppm
I20260811 00:13:05.704182 16921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:05 dist-test-slave-8hhm krb5kdc[16898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407185, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:05.707229 16921 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:13:05.708712 16921 webserver.cc:460] Webserver started at http://127.7.94.62:45791/ using document root <none> and password file <none>
I20260811 00:13:05.709462 16921 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:05.709609 16921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:05.709933 16921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:05.712874 16921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "b29b57d664054ce7b5eb93a94077ce65"
format_stamp: "Formatted at 2026-08-11 00:13:05 on dist-test-slave-8hhm"
server_key: "e7aa56dfd05e4d2909ce469929d79a44"
server_key_iv: "aa6cd357005cf628363dbf3417b6fe89"
server_key_version: "encryptionkey@0"
I20260811 00:13:05.713670 16921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "b29b57d664054ce7b5eb93a94077ce65"
format_stamp: "Formatted at 2026-08-11 00:13:05 on dist-test-slave-8hhm"
server_key: "e7aa56dfd05e4d2909ce469929d79a44"
server_key_iv: "aa6cd357005cf628363dbf3417b6fe89"
server_key_version: "encryptionkey@0"
I20260811 00:13:05.718642 16921 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 00:13:05.722004 16943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:05.723282 16921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:13:05.723465 16921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "b29b57d664054ce7b5eb93a94077ce65"
format_stamp: "Formatted at 2026-08-11 00:13:05 on dist-test-slave-8hhm"
server_key: "e7aa56dfd05e4d2909ce469929d79a44"
server_key_iv: "aa6cd357005cf628363dbf3417b6fe89"
server_key_version: "encryptionkey@0"
I20260811 00:13:05.723676 16921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:05.741916 16921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:05.746140 16921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:05.746424 16921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:05.768126 16921 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:45677
I20260811 00:13:05.768170 17031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:45677 every 8 connection(s)
I20260811 00:13:05.769552 16921 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:13:05.773299 17032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:05.775113 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 16921
I20260811 00:13:05.775326 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:13:05.776144 7544 external_mini_cluster.cc:1428] Setting key cd807cf5fa74670323e46cb303fdb06e
I20260811 00:13:05.788328 17032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65: Bootstrap starting.
Aug 11 00:13:05 dist-test-slave-8hhm krb5kdc[16898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:13:05.792120 17032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:05.793303 17032 log.cc:824] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65: Log is configured to *not* fsync() on all Append() calls
I20260811 00:13:05.796262 17032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65: No bootstrap required, opened a new log
I20260811 00:13:05.799116 17036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:05.778201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50254 (local address 127.7.94.62:45677)
0811 00:13:05.779022 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:05.779034 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:05.779081 (+ 47us) server_negotiation.cc:407] Connection header received
0811 00:13:05.780647 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:05.780668 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:05.781053 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:05.781449 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:05.783209 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:05.784467 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:05.786245 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:05.786898 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:05.787030 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:05.791832 (+ 4802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:05.791840 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:05.791854 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:05.791891 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:05.795081 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:05.795809 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:05.795816 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:05.795822 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:05.795989 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:05.796380 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:05.796392 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:05.796394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:05.796917 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:05.797289 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:05.797656 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 00:13:05.798116 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":383,"threads_started":1}
I20260811 00:13:05.804322 17032 raft_consensus.cc:348] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b29b57d664054ce7b5eb93a94077ce65" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45677 } }
I20260811 00:13:05.804714 17032 raft_consensus.cc:374] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:05.804826 17032 raft_consensus.cc:729] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b29b57d664054ce7b5eb93a94077ce65, State: Initialized, Role: FOLLOWER
I20260811 00:13:05.805436 17032 consensus_queue.cc:260] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b29b57d664054ce7b5eb93a94077ce65" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45677 } }
I20260811 00:13:05.805688 17032 raft_consensus.cc:388] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:13:05.805832 17032 raft_consensus.cc:482] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:13:05.806005 17032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:05.808522 17032 raft_consensus.cc:504] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b29b57d664054ce7b5eb93a94077ce65" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45677 } }
I20260811 00:13:05.809010 17032 leader_election.cc:302] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b29b57d664054ce7b5eb93a94077ce65; no voters:
I20260811 00:13:05.809675 17032 leader_election.cc:288] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:13:05.809808 17039 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:13:05.810539 17039 raft_consensus.cc:686] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65 [term 1 LEADER]: Becoming Leader. State: Replica: b29b57d664054ce7b5eb93a94077ce65, State: Running, Role: LEADER
I20260811 00:13:05.811075 17039 consensus_queue.cc:237] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b29b57d664054ce7b5eb93a94077ce65" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45677 } }
I20260811 00:13:05.811866 17032 sys_catalog.cc:564] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:13:05.813869 17044 sys_catalog.cc:455] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b29b57d664054ce7b5eb93a94077ce65. Latest consensus state: current_term: 1 leader_uuid: "b29b57d664054ce7b5eb93a94077ce65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b29b57d664054ce7b5eb93a94077ce65" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45677 } } }
I20260811 00:13:05.814354 17044 sys_catalog.cc:458] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65 [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:05.814337 17041 sys_catalog.cc:455] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b29b57d664054ce7b5eb93a94077ce65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b29b57d664054ce7b5eb93a94077ce65" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45677 } } }
I20260811 00:13:05.814659 17041 sys_catalog.cc:458] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65 [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:05.815972 17051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:13:05.821589 17051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:13:05.831133 17051 catalog_manager.cc:1347] Generated new cluster ID: c440b2859d744c2589890ab44fbb3d71
I20260811 00:13:05.831290 17051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:13:05.841145 17051 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:13:05.842482 17051 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:13:05.848312 17051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b29b57d664054ce7b5eb93a94077ce65: Generated new TSK 0
I20260811 00:13:05.849117 17051 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
I20260811 00:13:05.916474 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45677
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:46289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:06.257046 17082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:06.257426 17082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:06.257608 17082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:06.264892 17082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:06.265002 17082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:06.265256 17082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:13:06.274874 17082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:46289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:06.276268 17082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:06.277585 17082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:06.290684 17090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:06.290735 17094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:06.290709 17092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:06.291890 17082 server_base.cc:1034] running on GCE node
I20260811 00:13:06.292439 17082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:06.293359 17082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:06.294557 17082 hybrid_clock.cc:648] HybridClock initialized: now 1786407186294516 us; error 35 us; skew 500 ppm
I20260811 00:13:06.294837 17082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:06 dist-test-slave-8hhm krb5kdc[16898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407186, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:06.297976 17082 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:13:06.299496 17082 webserver.cc:460] Webserver started at http://127.7.94.1:38227/ using document root <none> and password file <none>
I20260811 00:13:06.300195 17082 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:06.300343 17082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:06.300745 17082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:06.303687 17082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "07cfe6b7fc6b432abb6121ffce780b30"
format_stamp: "Formatted at 2026-08-11 00:13:06 on dist-test-slave-8hhm"
server_key: "044a11718f49899e763c0bcb07036ecd"
server_key_iv: "b48b7a6f3e03d06a224eeb079fb90fb4"
server_key_version: "encryptionkey@0"
I20260811 00:13:06.304418 17082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "07cfe6b7fc6b432abb6121ffce780b30"
format_stamp: "Formatted at 2026-08-11 00:13:06 on dist-test-slave-8hhm"
server_key: "044a11718f49899e763c0bcb07036ecd"
server_key_iv: "b48b7a6f3e03d06a224eeb079fb90fb4"
server_key_version: "encryptionkey@0"
I20260811 00:13:06.309617 17082 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 00:13:06.312877 17107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:06.314210 17082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:13:06.314383 17082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "07cfe6b7fc6b432abb6121ffce780b30"
format_stamp: "Formatted at 2026-08-11 00:13:06 on dist-test-slave-8hhm"
server_key: "044a11718f49899e763c0bcb07036ecd"
server_key_iv: "b48b7a6f3e03d06a224eeb079fb90fb4"
server_key_version: "encryptionkey@0"
I20260811 00:13:06.314579 17082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:06.331876 17082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:06.335786 17082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:06.336244 17082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:06.337366 17082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:06.338800 17082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:06.338924 17082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:06.339099 17082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:06.339201 17082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:06.375212 17082 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:42819
I20260811 00:13:06.375268 17298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:42819 every 8 connection(s)
I20260811 00:13:06.376735 17082 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:13:06.382220 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 17082
I20260811 00:13:06.382433 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:13:06.383351 7544 external_mini_cluster.cc:1428] Setting key 2e603b5ba563a3b45c1621e12d2944e7
Aug 11 00:13:06 dist-test-slave-8hhm krb5kdc[16898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407186, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:13:06.400933 17303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:06.379406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:34053 (local address 127.7.94.62:45677)
0811 00:13:06.379848 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:06.379853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:06.380908 (+ 1055us) server_negotiation.cc:407] Connection header received
0811 00:13:06.383755 (+ 2847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:06.383761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:06.383862 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:06.383960 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:06.386110 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:06.386788 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:06.389652 (+ 2864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:06.390032 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:06.390122 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:06.395090 (+ 4968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:06.395095 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:06.395098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:06.395129 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:06.397853 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:06.398605 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:06.398612 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:06.398620 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:06.398734 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:06.399199 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:06.399203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:06.399205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:06.399465 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:06.399618 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:06.400350 (+ 732us) server_negotiation.cc:299] Negotiation successful
0811 00:13:06.400565 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":222,"threads_started":1}
I20260811 00:13:06.401969 17304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:06.379939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:45677 (local address 127.7.94.1:34053)
0811 00:13:06.380731 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:06.380779 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:06.383377 (+ 2598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:06.384267 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:06.384276 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:06.385019 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:06.385909 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:06.385938 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:06.386965 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:06.386972 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:06.389497 (+ 2525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:06.389516 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:06.390268 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:06.390278 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:06.390633 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:06.391390 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:06.391434 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:06.394831 (+ 3397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:06.398006 (+ 3175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:06.398014 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:06.398031 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:06.398470 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:06.398851 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:06.398856 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:06.398860 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:06.399054 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:06.399596 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:06.399604 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:06.399943 (+ 339us) client_negotiation.cc:769] Sending connection context
0811 00:13:06.400237 (+ 294us) client_negotiation.cc:240] Negotiation successful
0811 00:13:06.400771 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":337,"threads_started":1}
I20260811 00:13:06.403548 17301 heartbeater.cc:347] Connected to a master server at 127.7.94.62:45677
I20260811 00:13:06.403893 17301 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:06.404862 17301 heartbeater.cc:511] Master 127.7.94.62:45677 requested a full tablet report, sending...
I20260811 00:13:06.407423 16972 ts_manager.cc:194] Registered new tserver with Master: 07cfe6b7fc6b432abb6121ffce780b30 (127.7.94.1:42819)
I20260811 00:13:06.409691 16972 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:34053
WARNING: no policy specified for kudu/127.7.94.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
I20260811 00:13:06.442152 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45677
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:46289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:06.779944 17311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:06.780280 17311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:06.780419 17311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:06.791291 17311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:06.791409 17311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:06.791530 17311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:13:06.801151 17311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:46289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:06.802544 17311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:06.803867 17311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:06.817066 17327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:06.817028 17324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:06.817248 17320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:06.818216 17311 server_base.cc:1034] running on GCE node
I20260811 00:13:06.818823 17311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:06.819832 17311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:06.821055 17311 hybrid_clock.cc:648] HybridClock initialized: now 1786407186820993 us; error 53 us; skew 500 ppm
I20260811 00:13:06.821412 17311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:06 dist-test-slave-8hhm krb5kdc[16898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407186, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:06.824527 17311 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:13:06.826094 17311 webserver.cc:460] Webserver started at http://127.7.94.2:37555/ using document root <none> and password file <none>
I20260811 00:13:06.826809 17311 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:06.826980 17311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:06.827258 17311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:06.830224 17311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "99aebb191aee4eda9f49942d2f021e98"
format_stamp: "Formatted at 2026-08-11 00:13:06 on dist-test-slave-8hhm"
server_key: "ae944993a68542f0fa46b0c10d5b82c2"
server_key_iv: "d388d378d26937dba5b618b808fa7191"
server_key_version: "encryptionkey@0"
I20260811 00:13:06.830947 17311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "99aebb191aee4eda9f49942d2f021e98"
format_stamp: "Formatted at 2026-08-11 00:13:06 on dist-test-slave-8hhm"
server_key: "ae944993a68542f0fa46b0c10d5b82c2"
server_key_iv: "d388d378d26937dba5b618b808fa7191"
server_key_version: "encryptionkey@0"
I20260811 00:13:06.835985 17311 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:13:06.839290 17337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:06.840579 17311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:13:06.840770 17311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "99aebb191aee4eda9f49942d2f021e98"
format_stamp: "Formatted at 2026-08-11 00:13:06 on dist-test-slave-8hhm"
server_key: "ae944993a68542f0fa46b0c10d5b82c2"
server_key_iv: "d388d378d26937dba5b618b808fa7191"
server_key_version: "encryptionkey@0"
I20260811 00:13:06.840970 17311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:06.859020 17311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:06.863125 17311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:06.863451 17311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:06.864391 17311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:06.865865 17311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:06.865939 17311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:06.866113 17311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:06.866166 17311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:06.908324 17311 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:37953
I20260811 00:13:06.908360 17536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:37953 every 8 connection(s)
I20260811 00:13:06.909775 17311 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:13:06.916451 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 17311
I20260811 00:13:06.916626 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:13:06.917130 7544 external_mini_cluster.cc:1428] Setting key 84be63b98caf68dad06c9aeb2771a8e8
Aug 11 00:13:06 dist-test-slave-8hhm krb5kdc[16898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407186, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:13:06.931277 17543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:06.912497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:60367 (local address 127.7.94.62:45677)
0811 00:13:06.913056 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:06.913060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:06.914010 (+ 950us) server_negotiation.cc:407] Connection header received
0811 00:13:06.915915 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:06.915921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:06.916015 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:06.916155 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:06.918344 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:06.919221 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:06.920861 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:06.921395 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:06.921583 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:06.925574 (+ 3991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:06.925579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:06.925582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:06.925612 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:06.928250 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:06.928974 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:06.928980 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:06.928983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:06.929115 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:06.929593 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:06.929598 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:06.929603 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:06.929881 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:06.930043 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:06.930696 (+ 653us) server_negotiation.cc:299] Negotiation successful
0811 00:13:06.930905 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":324,"threads_started":1}
I20260811 00:13:06.932241 17544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:06.913000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:45677 (local address 127.7.94.2:60367)
0811 00:13:06.913811 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:06.913858 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:06.915632 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:06.916479 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:06.916487 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:06.917273 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:06.918101 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:06.918124 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:06.919395 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:06.919408 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:06.920678 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:06.920692 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:06.921673 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:06.921682 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:06.921922 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:06.922713 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:06.922745 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:06.925352 (+ 2607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:06.928394 (+ 3042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:06.928404 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:06.928420 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:06.928845 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:06.929266 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:06.929272 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:06.929278 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:06.929471 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:06.930008 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:06.930016 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:06.930349 (+ 333us) client_negotiation.cc:769] Sending connection context
0811 00:13:06.930603 (+ 254us) client_negotiation.cc:240] Negotiation successful
0811 00:13:06.931117 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":340,"threads_started":1}
I20260811 00:13:06.933727 17538 heartbeater.cc:347] Connected to a master server at 127.7.94.62:45677
I20260811 00:13:06.934074 17538 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:06.935016 17538 heartbeater.cc:511] Master 127.7.94.62:45677 requested a full tablet report, sending...
I20260811 00:13:06.936582 16972 ts_manager.cc:194] Registered new tserver with Master: 99aebb191aee4eda9f49942d2f021e98 (127.7.94.2:37953)
I20260811 00:13:06.937911 16972 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:60367
WARNING: no policy specified for kudu/127.7.94.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
I20260811 00:13:06.972783 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45677
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:46289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:07.301877 17555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:07.302157 17555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:07.302249 17555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:07.310070 17555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:07.310173 17555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:07.310289 17555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:13:07.320043 17555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:46289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:07.321298 17555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:07.322577 17555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:07.335724 17568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:07.335690 17570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:07.335776 17574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:07.336791 17555 server_base.cc:1034] running on GCE node
I20260811 00:13:07.337615 17555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:07.338658 17555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:07.339913 17555 hybrid_clock.cc:648] HybridClock initialized: now 1786407187339870 us; error 29 us; skew 500 ppm
I20260811 00:13:07.340307 17555 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:07 dist-test-slave-8hhm krb5kdc[16898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407187, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:07.343804 17555 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:13:07.345566 17555 webserver.cc:460] Webserver started at http://127.7.94.3:36021/ using document root <none> and password file <none>
I20260811 00:13:07.346304 17555 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:07.346460 17555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:07.346803 17555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:07.349996 17555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "b84a70b4c6514cd5abea0b4954006f0e"
format_stamp: "Formatted at 2026-08-11 00:13:07 on dist-test-slave-8hhm"
server_key: "b6de4dcb58fb15b59f67e8e9c9e4b4d8"
server_key_iv: "b0972a912e165b97e46a076844ce8294"
server_key_version: "encryptionkey@0"
I20260811 00:13:07.350850 17555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "b84a70b4c6514cd5abea0b4954006f0e"
format_stamp: "Formatted at 2026-08-11 00:13:07 on dist-test-slave-8hhm"
server_key: "b6de4dcb58fb15b59f67e8e9c9e4b4d8"
server_key_iv: "b0972a912e165b97e46a076844ce8294"
server_key_version: "encryptionkey@0"
I20260811 00:13:07.356269 17555 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 00:13:07.359663 17585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:07.360867 17555 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:13:07.361064 17555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "b84a70b4c6514cd5abea0b4954006f0e"
format_stamp: "Formatted at 2026-08-11 00:13:07 on dist-test-slave-8hhm"
server_key: "b6de4dcb58fb15b59f67e8e9c9e4b4d8"
server_key_iv: "b0972a912e165b97e46a076844ce8294"
server_key_version: "encryptionkey@0"
I20260811 00:13:07.361292 17555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:07.391811 17555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:07.396363 17555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:07.396602 17555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:07.397567 17555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:07.399031 17555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:07.399133 17555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:07.399264 17555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:07.399363 17555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:07.413105 17301 heartbeater.cc:503] Master 127.7.94.62:45677 was elected leader, sending a full tablet report...
I20260811 00:13:07.434273 17555 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:34907
I20260811 00:13:07.434361 17775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:34907 every 8 connection(s)
I20260811 00:13:07.435575 17555 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:13:07.445962 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 17555
I20260811 00:13:07.446156 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:13:07.446620 7544 external_mini_cluster.cc:1428] Setting key 9cf467e172d13f9fb54dc2c3e3ce9ef2
Aug 11 00:13:07 dist-test-slave-8hhm krb5kdc[16898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407187, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:13:07.455427 17779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:07.438058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:33903 (local address 127.7.94.62:45677)
0811 00:13:07.438558 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:07.438563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:07.439394 (+ 831us) server_negotiation.cc:407] Connection header received
0811 00:13:07.441088 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:07.441094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:07.441185 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:07.441358 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:07.442799 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:07.443520 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:07.445396 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:07.446056 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:07.446192 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:07.450179 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:07.450183 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:07.450186 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:07.450217 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:07.452733 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:07.453521 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:07.453525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:07.453527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:07.453642 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:07.454030 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:07.454033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:07.454035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:07.454281 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:07.454393 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:07.455010 (+ 617us) server_negotiation.cc:299] Negotiation successful
0811 00:13:07.455171 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":279,"threads_started":1}
I20260811 00:13:07.456355 17781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:07.438530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:45677 (local address 127.7.94.3:33903)
0811 00:13:07.439229 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:07.439267 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:07.440824 (+ 1557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:07.441626 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:07.441634 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:07.442096 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:07.442618 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:07.442634 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:07.443682 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:07.443686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:07.445202 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:07.445257 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:07.446320 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:07.446327 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:07.446524 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:07.447245 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:07.447266 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:07.450012 (+ 2746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:07.452880 (+ 2868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:07.452886 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:07.452898 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:07.453412 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:07.453745 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:07.453748 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:07.453750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:07.453930 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:07.454390 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:07.454395 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:07.454698 (+ 303us) client_negotiation.cc:769] Sending connection context
0811 00:13:07.454914 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 00:13:07.455288 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":298,"threads_started":1}
I20260811 00:13:07.457787 17776 heartbeater.cc:347] Connected to a master server at 127.7.94.62:45677
I20260811 00:13:07.458086 17776 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:07.458813 17776 heartbeater.cc:511] Master 127.7.94.62:45677 requested a full tablet report, sending...
I20260811 00:13:07.460215 16972 ts_manager.cc:194] Registered new tserver with Master: b84a70b4c6514cd5abea0b4954006f0e (127.7.94.3:34907)
I20260811 00:13:07.461355 16972 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:33903
I20260811 00:13:07.462270 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/test-admin.keytab.
Aug 11 00:13:07 dist-test-slave-8hhm krb5kdc[16898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:07.494915 7544 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 00:13:07.500283 17792 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:07.497607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45896 (local address 127.7.94.1:42819)
0811 00:13:07.498271 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:07.498280 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:07.498313 (+ 33us) server_negotiation.cc:407] Connection header received
0811 00:13:07.498508 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:07.498515 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:07.498735 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:07.498905 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:07.499864 (+ 959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45896: BlockingRecv error: recv got EOF from 127.0.0.1:45896 (error 108)
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":289,"threads_started":1}
W20260811 00:13:07.500428 7544 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/krb5cc' of type FILE
Aug 11 00:13:07 dist-test-slave-8hhm krb5kdc[16898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:07.502032 7544 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 00:13:07 dist-test-slave-8hhm krb5kdc[16898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:07.503379 7544 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 00:13:07 dist-test-slave-8hhm krb5kdc[16898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:13:07.518340 17792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:07.505028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45900 (local address 127.7.94.1:42819)
0811 00:13:07.505207 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:07.505233 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:07.505435 (+ 202us) server_negotiation.cc:407] Connection header received
0811 00:13:07.505572 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:07.505576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:07.505627 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:07.505713 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:07.506633 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:07.507569 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:07.508749 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:07.509381 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:07.509475 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:07.513039 (+ 3564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:07.513050 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:07.513064 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:07.513093 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:07.515724 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:07.516289 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:07.516295 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:07.516297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:07.516463 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:07.516814 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:07.516817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:07.516818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:07.517189 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:07.517446 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:07.517781 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 00:13:07.518064 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 00:13:07.524539 17792 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:07.522597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45902 (local address 127.7.94.1:42819)
0811 00:13:07.522797 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:07.522804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:07.523044 (+ 240us) server_negotiation.cc:407] Connection header received
0811 00:13:07.523228 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:07.523234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:07.523313 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:07.523418 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:07.524267 (+ 849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45902: BlockingRecv error: recv got EOF from 127.0.0.1:45902 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 00:13:07.524753 7544 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/krb5cc' of type FILE
Aug 11 00:13:07 dist-test-slave-8hhm krb5kdc[16898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:07.526286 7544 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 00:13:07 dist-test-slave-8hhm krb5kdc[16898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:07.527678 7544 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 00:13:07 dist-test-slave-8hhm krb5kdc[16898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:13:07.540974 17792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:07.529026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45908 (local address 127.7.94.1:42819)
0811 00:13:07.529198 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:07.529204 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:07.529454 (+ 250us) server_negotiation.cc:407] Connection header received
0811 00:13:07.529613 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:07.529617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:07.529666 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:07.529740 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:07.530735 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:07.531418 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:07.532572 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:07.532981 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:07.533067 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:07.536412 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:07.536417 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:07.536420 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:07.536462 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:07.538880 (+ 2418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:07.539429 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:07.539433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:07.539435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:07.539549 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:07.539866 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:07.539869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:07.539871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:07.540109 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:07.540298 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:07.540517 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 00:13:07.540715 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 00:13:07.545425 17792 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:07.543445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45914 (local address 127.7.94.1:42819)
0811 00:13:07.543586 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:07.543589 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:07.543971 (+ 382us) server_negotiation.cc:407] Connection header received
0811 00:13:07.544088 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:07.544091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:07.544162 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:07.544256 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:07.545133 (+ 877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45914: BlockingRecv error: recv got EOF from 127.0.0.1:45914 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 00:13:07.545440 7544 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786407133955383-7544-0/krb5kdc/krb5cc' of type FILE
Aug 11 00:13:07 dist-test-slave-8hhm krb5kdc[16898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:07.546969 7544 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 00:13:07 dist-test-slave-8hhm krb5kdc[16898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:07.548226 7544 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 00:13:07 dist-test-slave-8hhm krb5kdc[16898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:13:07.561421 17792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:07.549390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45920 (local address 127.7.94.1:42819)
0811 00:13:07.549594 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:07.549600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:07.549823 (+ 223us) server_negotiation.cc:407] Connection header received
0811 00:13:07.549914 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:07.549917 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:07.549968 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:07.550064 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:07.550893 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:07.551609 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:07.552771 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:07.553171 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:07.553391 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:07.556647 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:07.556652 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:07.556655 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:07.556683 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:07.559175 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:07.559722 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:07.559726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:07.559728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:07.559872 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:07.560174 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:07.560177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:07.560179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:07.560444 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:07.560671 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:07.560826 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 00:13:07.560978 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 00:13:07.562852 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 17082
I20260811 00:13:07.573989 17792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:07.563528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45934 (local address 127.7.94.1:42819)
0811 00:13:07.563661 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:07.563665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:07.564110 (+ 445us) server_negotiation.cc:407] Connection header received
0811 00:13:07.564232 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:07.564235 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:07.564282 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:07.564361 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:07.565372 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:07.566109 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:07.567209 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:07.567624 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:07.567711 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:07.569645 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:07.569650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:07.569653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:07.569679 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:07.571925 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:07.572479 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:07.572482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:07.572484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:07.572592 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:07.572902 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:07.572905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:07.572907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:07.573148 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:07.573430 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:07.573616 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 00:13:07.573763 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 00:13:07.574517 17293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:07.657888 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 17082
I20260811 00:13:07.689903 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 17311
Aug 11 00:13:07 dist-test-slave-8hhm krb5kdc[16898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.2@KRBTEST.COM
I20260811 00:13:07.704982 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:07.690869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58590 (local address 127.7.94.2:37953)
0811 00:13:07.691465 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:07.691472 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:07.691498 (+ 26us) server_negotiation.cc:407] Connection header received
0811 00:13:07.691559 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:07.691565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:07.691809 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:07.691945 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:07.692849 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:07.693959 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:07.695121 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:07.695639 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:07.695734 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:07.699624 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:07.699636 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:07.699650 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:07.699685 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:07.702583 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:07.703208 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:07.703215 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:07.703217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:07.703340 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:07.703639 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:07.703642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:07.703644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:07.703941 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:07.704157 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:07.704414 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 00:13:07.704670 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":314,"threads_started":1}
I20260811 00:13:07.706287 17528 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:07.791956 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 17311
I20260811 00:13:07.818773 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 17555
Aug 11 00:13:07 dist-test-slave-8hhm krb5kdc[16898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.3@KRBTEST.COM
I20260811 00:13:07.834671 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:07.819740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38332 (local address 127.7.94.3:34907)
0811 00:13:07.820387 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:07.820393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:07.820422 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:13:07.820488 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:07.820494 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:07.820702 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:07.820882 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:07.822057 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:07.823122 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:07.824392 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:07.824954 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:07.825116 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:07.829172 (+ 4056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:07.829183 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:07.829196 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:07.829274 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:07.831914 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:07.832575 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:07.832591 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:07.832594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:07.832716 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:07.833119 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:07.833128 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:07.833130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:07.833506 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:07.833829 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:07.834115 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 00:13:07.834364 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":325,"threads_started":1}
I20260811 00:13:07.835974 17767 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:07.919130 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 17555
I20260811 00:13:07.943244 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 16921
I20260811 00:13:07.944470 17022 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:08.047709 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 16921
2026-08-11T00:13:08Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4453 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:13:08 dist-test-slave-8hhm krb5kdc[17825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:13:08 dist-test-slave-8hhm krb5kdc[17825](info): set up 2 sockets
Aug 11 00:13:08 dist-test-slave-8hhm krb5kdc[17825](info): commencing operation
krb5kdc: starting...
W20260811 00:13:09.380347 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:13:09 dist-test-slave-8hhm krb5kdc[17825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:13:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:13:09Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.62.keytab.
Entry for principal oryx/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.62.keytab.
I20260811 00:13:09.604424 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:36587
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.62.keytab
--principal=oryx/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:39205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:36587
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:09.922920 17852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:09.923172 17852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:09.923341 17852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:09.930275 17852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:13:09.930371 17852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:09.930469 17852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:09.930567 17852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:13:09.930642 17852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:13:09.939993 17852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:39205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:36587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.62.keytab
--principal=oryx/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:36587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:09.941188 17852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:09.942464 17852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:09.954303 17858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:09.954392 17859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:09.954342 17861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:09.955148 17852 server_base.cc:1034] running on GCE node
I20260811 00:13:09.955812 17852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:09.957038 17852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:09.958240 17852 hybrid_clock.cc:648] HybridClock initialized: now 1786407189958195 us; error 37 us; skew 500 ppm
I20260811 00:13:09.958607 17852 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:09 dist-test-slave-8hhm krb5kdc[17825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407189, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:09.965677 17852 init.cc:377] Logged in from keytab as oryx/127.7.94.62@KRBTEST.COM (short username oryx)
I20260811 00:13:09.967123 17852 webserver.cc:460] Webserver started at http://127.7.94.62:39095/ using document root <none> and password file <none>
I20260811 00:13:09.967782 17852 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:09.967921 17852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:09.968217 17852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:09.971153 17852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "aad6ba1a2f694580b8fa3dc892b1d83b"
format_stamp: "Formatted at 2026-08-11 00:13:09 on dist-test-slave-8hhm"
server_key: "335b95ecc3a3fa9af399501972a539e5"
server_key_iv: "88c2d6acce7a15e9af1bc42769fe6615"
server_key_version: "encryptionkey@0"
I20260811 00:13:09.971925 17852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "aad6ba1a2f694580b8fa3dc892b1d83b"
format_stamp: "Formatted at 2026-08-11 00:13:09 on dist-test-slave-8hhm"
server_key: "335b95ecc3a3fa9af399501972a539e5"
server_key_iv: "88c2d6acce7a15e9af1bc42769fe6615"
server_key_version: "encryptionkey@0"
I20260811 00:13:09.976953 17852 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 00:13:09.980257 17877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:09.981572 17852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:13:09.981762 17852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "aad6ba1a2f694580b8fa3dc892b1d83b"
format_stamp: "Formatted at 2026-08-11 00:13:09 on dist-test-slave-8hhm"
server_key: "335b95ecc3a3fa9af399501972a539e5"
server_key_iv: "88c2d6acce7a15e9af1bc42769fe6615"
server_key_version: "encryptionkey@0"
I20260811 00:13:09.981953 17852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:10.006145 17852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:10.010046 17852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:10.010349 17852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:10.028534 17852 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:36587
I20260811 00:13:10.028565 17975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:36587 every 8 connection(s)
I20260811 00:13:10.029923 17852 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:13:10.032948 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 17852
I20260811 00:13:10.033200 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:13:10.033565 17977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:10.033911 7544 external_mini_cluster.cc:1428] Setting key 1971bfc6e989d0b0d9b37a33588f13cf
I20260811 00:13:10.047426 17977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b: Bootstrap starting.
Aug 11 00:13:10 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM
I20260811 00:13:10.050532 17977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:10.051553 17977 log.cc:824] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b: Log is configured to *not* fsync() on all Append() calls
I20260811 00:13:10.054416 17977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b: No bootstrap required, opened a new log
I20260811 00:13:10.056424 17981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:10.035773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41510 (local address 127.7.94.62:36587)
0811 00:13:10.036597 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:10.036611 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:10.036649 (+ 38us) server_negotiation.cc:407] Connection header received
0811 00:13:10.038259 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:10.038282 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:10.038665 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:10.038978 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:10.040839 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:10.042189 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:10.043974 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:10.044635 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:10.044774 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:10.049354 (+ 4580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:10.049363 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:10.049376 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:10.049411 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:10.052488 (+ 3077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:10.053190 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:10.053196 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:10.053202 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:10.053414 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:10.053832 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:10.053837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:10.053842 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:10.054328 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:10.054645 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:10.055092 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 00:13:10.055502 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":414,"threads_started":1}
I20260811 00:13:10.062485 17977 raft_consensus.cc:348] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } }
I20260811 00:13:10.062877 17977 raft_consensus.cc:374] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:10.063019 17977 raft_consensus.cc:729] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aad6ba1a2f694580b8fa3dc892b1d83b, State: Initialized, Role: FOLLOWER
I20260811 00:13:10.063586 17977 consensus_queue.cc:260] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } }
I20260811 00:13:10.063848 17977 raft_consensus.cc:388] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:13:10.063984 17977 raft_consensus.cc:482] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:13:10.064167 17977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:10.066645 17977 raft_consensus.cc:504] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } }
I20260811 00:13:10.067117 17977 leader_election.cc:302] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aad6ba1a2f694580b8fa3dc892b1d83b; no voters:
I20260811 00:13:10.067699 17977 leader_election.cc:288] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:13:10.067855 17988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:13:10.068542 17988 raft_consensus.cc:686] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [term 1 LEADER]: Becoming Leader. State: Replica: aad6ba1a2f694580b8fa3dc892b1d83b, State: Running, Role: LEADER
I20260811 00:13:10.069077 17988 consensus_queue.cc:237] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } }
I20260811 00:13:10.069846 17977 sys_catalog.cc:564] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:13:10.071762 17991 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } }
I20260811 00:13:10.072116 17991 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:10.073518 17992 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: SysCatalogTable state changed. Reason: New leader aad6ba1a2f694580b8fa3dc892b1d83b. Latest consensus state: current_term: 1 leader_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } }
I20260811 00:13:10.073987 17992 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:10.075467 18000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:13:10.080761 18000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:13:10.089097 18000 catalog_manager.cc:1347] Generated new cluster ID: 98709dcf80214cd5a07efe330d81c3d5
I20260811 00:13:10.089202 18000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:13:10.109946 18000 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:13:10.111642 18000 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:13:10.120465 18000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b: Generated new TSK 0
I20260811 00:13:10.121392 18000 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.1.keytab.
Entry for principal oryx/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.1.keytab.
I20260811 00:13:10.176748 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:36587
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.1.keytab
--principal=oryx/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:39205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:10.508589 18029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:10.508919 18029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:10.509110 18029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:10.516198 18029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:10.516347 18029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:10.516530 18029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:13:10.526118 18029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:39205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.1.keytab
--principal=oryx/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:36587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:10.527398 18029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:10.528848 18029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:10.542229 18042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:10.542187 18041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:10.542320 18050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:10.543246 18029 server_base.cc:1034] running on GCE node
I20260811 00:13:10.543869 18029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:10.544806 18029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:10.546018 18029 hybrid_clock.cc:648] HybridClock initialized: now 1786407190545980 us; error 33 us; skew 500 ppm
I20260811 00:13:10.546339 18029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:10 dist-test-slave-8hhm krb5kdc[17825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407190, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:10.549670 18029 init.cc:377] Logged in from keytab as oryx/127.7.94.1@KRBTEST.COM (short username oryx)
I20260811 00:13:10.550982 18029 webserver.cc:460] Webserver started at http://127.7.94.1:34911/ using document root <none> and password file <none>
I20260811 00:13:10.551582 18029 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:10.551689 18029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:10.551952 18029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:10.554749 18029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "c488dd19f66d45419d4054c024614935"
format_stamp: "Formatted at 2026-08-11 00:13:10 on dist-test-slave-8hhm"
server_key: "ee25d3eb77d13be02b357fc27442f6d1"
server_key_iv: "7e9308363e11f173fe9967d94eb99240"
server_key_version: "encryptionkey@0"
I20260811 00:13:10.555444 18029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "c488dd19f66d45419d4054c024614935"
format_stamp: "Formatted at 2026-08-11 00:13:10 on dist-test-slave-8hhm"
server_key: "ee25d3eb77d13be02b357fc27442f6d1"
server_key_iv: "7e9308363e11f173fe9967d94eb99240"
server_key_version: "encryptionkey@0"
I20260811 00:13:10.560442 18029 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 00:13:10.563680 18062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:10.564958 18029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:13:10.565145 18029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "c488dd19f66d45419d4054c024614935"
format_stamp: "Formatted at 2026-08-11 00:13:10 on dist-test-slave-8hhm"
server_key: "ee25d3eb77d13be02b357fc27442f6d1"
server_key_iv: "7e9308363e11f173fe9967d94eb99240"
server_key_version: "encryptionkey@0"
I20260811 00:13:10.565400 18029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:10.584347 18029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:10.588228 18029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:10.588517 18029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:10.589452 18029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:10.590689 18029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:10.590787 18029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:10.590932 18029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:10.591001 18029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:10.623576 18029 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:36059
I20260811 00:13:10.623664 18272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:36059 every 8 connection(s)
I20260811 00:13:10.624794 18029 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:13:10.628440 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 18029
I20260811 00:13:10.628654 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:13:10.629426 7544 external_mini_cluster.cc:1428] Setting key c40ff9c15dfb11ca011f55e85e68dcfb
Aug 11 00:13:10 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407190, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM
I20260811 00:13:10.645630 18276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:10.627319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:51001 (local address 127.7.94.62:36587)
0811 00:13:10.627760 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:10.627765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:10.628631 (+ 866us) server_negotiation.cc:407] Connection header received
0811 00:13:10.630642 (+ 2011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:10.630647 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:10.630744 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:10.630861 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:10.632305 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:10.633008 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:10.634609 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:10.635007 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:10.635141 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:10.639130 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:10.639135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:10.639138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:10.639178 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:10.642364 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:10.643081 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:10.643089 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:10.643096 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:10.643247 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:10.643642 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:10.643658 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:10.643664 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:10.644027 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:10.644179 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:10.644879 (+ 700us) server_negotiation.cc:299] Negotiation successful
0811 00:13:10.645123 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":244,"threads_started":1}
I20260811 00:13:10.646153 18278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:10.627713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.1:51001)
0811 00:13:10.628423 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:10.628461 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:10.630368 (+ 1907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:10.631102 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:10.631110 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:10.631586 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:10.632138 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:10.632157 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:10.633161 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:10.633166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:10.634442 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:10.634463 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:10.635240 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:10.635249 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:10.635479 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:10.636166 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:10.636188 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:10.638914 (+ 2726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:10.642522 (+ 3608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:10.642528 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:10.642539 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:10.642952 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:10.643351 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:10.643354 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:10.643356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:10.643527 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:10.644147 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:10.644152 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:10.644435 (+ 283us) client_negotiation.cc:769] Sending connection context
0811 00:13:10.644669 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 00:13:10.645049 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":319,"threads_started":1}
I20260811 00:13:10.647590 18274 heartbeater.cc:347] Connected to a master server at 127.7.94.62:36587
I20260811 00:13:10.647927 18274 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:10.648855 18274 heartbeater.cc:511] Master 127.7.94.62:36587 requested a full tablet report, sending...
I20260811 00:13:10.651005 17914 ts_manager.cc:194] Registered new tserver with Master: c488dd19f66d45419d4054c024614935 (127.7.94.1:36059)
I20260811 00:13:10.652491 17914 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:51001
WARNING: no policy specified for oryx/127.7.94.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.94.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.2.keytab.
Entry for principal oryx/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.2.keytab.
I20260811 00:13:10.686625 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:36587
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.2.keytab
--principal=oryx/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:39205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:11.016104 18282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:11.016395 18282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:11.016559 18282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:11.023423 18282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:11.023559 18282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:11.023741 18282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:13:11.032990 18282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:39205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.2.keytab
--principal=oryx/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:36587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:11.034204 18282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:11.035462 18282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:11.047729 18292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:11.047770 18288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:11.047866 18295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:11.048995 18282 server_base.cc:1034] running on GCE node
I20260811 00:13:11.049619 18282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:11.050506 18282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:11.051769 18282 hybrid_clock.cc:648] HybridClock initialized: now 1786407191051726 us; error 48 us; skew 500 ppm
I20260811 00:13:11.052102 18282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:11 dist-test-slave-8hhm krb5kdc[17825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407191, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:11.055305 18282 init.cc:377] Logged in from keytab as oryx/127.7.94.2@KRBTEST.COM (short username oryx)
I20260811 00:13:11.056618 18282 webserver.cc:460] Webserver started at http://127.7.94.2:39273/ using document root <none> and password file <none>
I20260811 00:13:11.057296 18282 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:11.057435 18282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:11.057794 18282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:11.060523 18282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "6c6905d72f434b7f86d5d92f6a46184c"
format_stamp: "Formatted at 2026-08-11 00:13:11 on dist-test-slave-8hhm"
server_key: "651738787c92206177f544ada2bfcb3c"
server_key_iv: "8f9958243e6fb031b4d9c6db66bd4e64"
server_key_version: "encryptionkey@0"
I20260811 00:13:11.061272 18282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "6c6905d72f434b7f86d5d92f6a46184c"
format_stamp: "Formatted at 2026-08-11 00:13:11 on dist-test-slave-8hhm"
server_key: "651738787c92206177f544ada2bfcb3c"
server_key_iv: "8f9958243e6fb031b4d9c6db66bd4e64"
server_key_version: "encryptionkey@0"
I20260811 00:13:11.066165 18282 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 00:13:11.069550 18306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:11.070948 18282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:13:11.071130 18282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "6c6905d72f434b7f86d5d92f6a46184c"
format_stamp: "Formatted at 2026-08-11 00:13:11 on dist-test-slave-8hhm"
server_key: "651738787c92206177f544ada2bfcb3c"
server_key_iv: "8f9958243e6fb031b4d9c6db66bd4e64"
server_key_version: "encryptionkey@0"
I20260811 00:13:11.071327 18282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:11.102392 18282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:11.106644 18282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:11.106876 18282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:11.107741 18282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:11.109064 18282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:11.109138 18282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:11.109246 18282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:11.109293 18282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 00:13:11.142046 18282 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:35337
I20260811 00:13:11.142095 18504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:35337 every 8 connection(s)
I20260811 00:13:11.143379 18282 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:13:11.149744 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 18282
I20260811 00:13:11.149941 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:13:11.150496 7544 external_mini_cluster.cc:1428] Setting key 4f3d125256b80a4b5ddf6e878895e116
Aug 11 00:13:11 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407191, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM
I20260811 00:13:11.164525 18276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:11.145987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:38297 (local address 127.7.94.62:36587)
0811 00:13:11.146205 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:11.146211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:11.147418 (+ 1207us) server_negotiation.cc:407] Connection header received
0811 00:13:11.149173 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:11.149178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:11.149307 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:11.149393 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:11.151419 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:11.152463 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:11.154061 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:11.154471 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:11.154611 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:11.157979 (+ 3368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:11.157984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:11.157987 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:11.158025 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:11.160525 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:11.161272 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:11.161278 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:11.161285 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:11.161420 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:11.161954 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:11.161962 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:11.161967 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:11.162355 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:11.162520 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:11.163477 (+ 957us) server_negotiation.cc:299] Negotiation successful
0811 00:13:11.163850 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":70}
I20260811 00:13:11.165491 18508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:11.146538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.2:38297)
0811 00:13:11.147259 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:11.147309 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:11.148922 (+ 1613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:11.149688 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:11.149702 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:11.150481 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:11.151234 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:11.151255 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:11.152623 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:11.152629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:11.153910 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:11.153923 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:11.154704 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:11.154711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:11.154927 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:11.155604 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:11.155625 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:11.157761 (+ 2136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:11.160702 (+ 2941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:11.160710 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:11.160724 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:11.161134 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:11.161537 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:11.161542 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:11.161546 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:11.161810 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:11.162512 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:11.162520 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:11.162944 (+ 424us) client_negotiation.cc:769] Sending connection context
0811 00:13:11.163270 (+ 326us) client_negotiation.cc:240] Negotiation successful
0811 00:13:11.163844 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":323,"threads_started":1}
I20260811 00:13:11.167186 18505 heartbeater.cc:347] Connected to a master server at 127.7.94.62:36587
I20260811 00:13:11.167547 18505 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:11.168574 18505 heartbeater.cc:511] Master 127.7.94.62:36587 requested a full tablet report, sending...
I20260811 00:13:11.170410 17914 ts_manager.cc:194] Registered new tserver with Master: 6c6905d72f434b7f86d5d92f6a46184c (127.7.94.2:35337)
I20260811 00:13:11.171692 17914 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:38297
WARNING: no policy specified for oryx/127.7.94.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.94.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.3.keytab.
Entry for principal oryx/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.3.keytab.
I20260811 00:13:11.215901 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:36587
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.3.keytab
--principal=oryx/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:39205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:11.549108 18511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:11.549439 18511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:11.549635 18511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:11.557071 18511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:11.557173 18511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:11.557385 18511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:13:11.566774 18511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:39205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.3.keytab
--principal=oryx/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:36587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:11.568127 18511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:11.569830 18511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:11.583026 18528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:11.583020 18531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:11.583091 18534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:11.583885 18511 server_base.cc:1034] running on GCE node
I20260811 00:13:11.584462 18511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:11.585480 18511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:11.586755 18511 hybrid_clock.cc:648] HybridClock initialized: now 1786407191586701 us; error 54 us; skew 500 ppm
I20260811 00:13:11.587085 18511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:11 dist-test-slave-8hhm krb5kdc[17825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407191, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:11.590268 18511 init.cc:377] Logged in from keytab as oryx/127.7.94.3@KRBTEST.COM (short username oryx)
I20260811 00:13:11.591569 18511 webserver.cc:460] Webserver started at http://127.7.94.3:46361/ using document root <none> and password file <none>
I20260811 00:13:11.592264 18511 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:11.592406 18511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:11.592705 18511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:11.595522 18511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "707ae6eda37543b087110d357cceadb8"
format_stamp: "Formatted at 2026-08-11 00:13:11 on dist-test-slave-8hhm"
server_key: "7eef5243ed906329bf38f5c885385988"
server_key_iv: "2576e09d1dd4d7a7e6a4fddd2957696a"
server_key_version: "encryptionkey@0"
I20260811 00:13:11.596269 18511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "707ae6eda37543b087110d357cceadb8"
format_stamp: "Formatted at 2026-08-11 00:13:11 on dist-test-slave-8hhm"
server_key: "7eef5243ed906329bf38f5c885385988"
server_key_iv: "2576e09d1dd4d7a7e6a4fddd2957696a"
server_key_version: "encryptionkey@0"
I20260811 00:13:11.601280 18511 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 00:13:11.604501 18550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:11.605734 18511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 00:13:11.605902 18511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "707ae6eda37543b087110d357cceadb8"
format_stamp: "Formatted at 2026-08-11 00:13:11 on dist-test-slave-8hhm"
server_key: "7eef5243ed906329bf38f5c885385988"
server_key_iv: "2576e09d1dd4d7a7e6a4fddd2957696a"
server_key_version: "encryptionkey@0"
I20260811 00:13:11.606091 18511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:11.619445 18511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:11.624204 18511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:11.624502 18511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:11.625552 18511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:11.626930 18511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:11.627036 18511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:11.627190 18511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:11.627276 18511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:11.656216 18274 heartbeater.cc:503] Master 127.7.94.62:36587 was elected leader, sending a full tablet report...
I20260811 00:13:11.660290 18511 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:34285
I20260811 00:13:11.660336 18736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:34285 every 8 connection(s)
I20260811 00:13:11.661646 18511 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:13:11.669638 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 18511
I20260811 00:13:11.669842 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:13:11.670403 7544 external_mini_cluster.cc:1428] Setting key 54c57869c7ba49039512dfe2af1273a2
Aug 11 00:13:11 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407191, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM
I20260811 00:13:11.681923 18276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:11.664138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:58973 (local address 127.7.94.62:36587)
0811 00:13:11.664303 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:11.664307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:11.665442 (+ 1135us) server_negotiation.cc:407] Connection header received
0811 00:13:11.667260 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:11.667265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:11.667352 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:11.667474 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:11.669017 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:11.670219 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:11.672155 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:11.672705 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:11.672842 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:11.676521 (+ 3679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:11.676527 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:11.676538 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:11.676569 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:11.678922 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:11.679639 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:11.679645 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:11.679648 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:11.679796 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:11.680218 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:11.680223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:11.680226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:11.680476 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:11.680639 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:11.681353 (+ 714us) server_negotiation.cc:299] Negotiation successful
0811 00:13:11.681567 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 00:13:11.682770 18742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:11.664558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.3:58973)
0811 00:13:11.665286 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:11.665330 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:11.666996 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:11.667720 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:11.667729 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:11.668236 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:11.668844 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:11.668861 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:11.670359 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:11.670363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:11.671990 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:11.672002 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:11.672955 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:11.672962 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:11.673298 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:11.673920 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:11.673940 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:11.676296 (+ 2356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:11.679076 (+ 2780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:11.679085 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:11.679107 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:11.679512 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:11.679907 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:11.679912 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:11.679916 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:11.680106 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:11.680603 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:11.680611 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:11.680914 (+ 303us) client_negotiation.cc:769] Sending connection context
0811 00:13:11.681203 (+ 289us) client_negotiation.cc:240] Negotiation successful
0811 00:13:11.681730 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":266,"threads_started":1}
I20260811 00:13:11.684098 18738 heartbeater.cc:347] Connected to a master server at 127.7.94.62:36587
I20260811 00:13:11.684458 18738 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:11.685467 18738 heartbeater.cc:511] Master 127.7.94.62:36587 requested a full tablet report, sending...
I20260811 00:13:11.686853 17914 ts_manager.cc:194] Registered new tserver with Master: 707ae6eda37543b087110d357cceadb8 (127.7.94.3:34285)
I20260811 00:13:11.687666 17914 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:58973
I20260811 00:13:11.697701 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.94.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.94.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.94.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.61.keytab.
Entry for principal oryx/127.7.94.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.61.keytab.
I20260811 00:13:11.752532 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.61:37589
--webserver_interface=127.7.94.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.61.keytab
--principal=oryx/127.7.94.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:39205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:36587,127.7.94.61:37589
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:12.094695 18747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:12.095009 18747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:12.095146 18747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:12.102406 18747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:13:12.102536 18747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:12.102646 18747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:12.102752 18747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:13:12.102841 18747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:13:12.112850 18747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:39205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:36587,127.7.94.61:37589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.61.keytab
--principal=oryx/127.7.94.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.61:37589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.94.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
W20260811 00:13:12.113328 18747 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.94.62:36587,127.7.94.61:37589'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 00:13:12.114395 18747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:12.115727 18747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:12.127385 18753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:12.127480 18752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:12.127501 18757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:12.128371 18747 server_base.cc:1034] running on GCE node
I20260811 00:13:12.129194 18747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:12.130527 18747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:12.131781 18747 hybrid_clock.cc:648] HybridClock initialized: now 1786407192131709 us; error 46 us; skew 500 ppm
I20260811 00:13:12.132099 18747 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:12 dist-test-slave-8hhm krb5kdc[17825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407192, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:12.135298 18747 init.cc:377] Logged in from keytab as oryx/127.7.94.61@KRBTEST.COM (short username oryx)
I20260811 00:13:12.136659 18747 webserver.cc:460] Webserver started at http://127.7.94.61:44203/ using document root <none> and password file <none>
I20260811 00:13:12.137411 18747 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:12.137539 18747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:12.137861 18747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:12.140782 18747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/data/instance:
uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4"
format_stamp: "Formatted at 2026-08-11 00:13:12 on dist-test-slave-8hhm"
server_key: "60b618afe60a805501f522a7bf7d1368"
server_key_iv: "9eb1337adbd2cbbd960ac3c8fb0680a7"
server_key_version: "encryptionkey@0"
I20260811 00:13:12.141698 18747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/wal/instance:
uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4"
format_stamp: "Formatted at 2026-08-11 00:13:12 on dist-test-slave-8hhm"
server_key: "60b618afe60a805501f522a7bf7d1368"
server_key_iv: "9eb1337adbd2cbbd960ac3c8fb0680a7"
server_key_version: "encryptionkey@0"
I20260811 00:13:12.146915 18747 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 00:13:12.150233 18764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:12.151418 18747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:13:12.151584 18747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/wal
uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4"
format_stamp: "Formatted at 2026-08-11 00:13:12 on dist-test-slave-8hhm"
server_key: "60b618afe60a805501f522a7bf7d1368"
server_key_iv: "9eb1337adbd2cbbd960ac3c8fb0680a7"
server_key_version: "encryptionkey@0"
I20260811 00:13:12.151813 18747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:12.175138 18505 heartbeater.cc:503] Master 127.7.94.62:36587 was elected leader, sending a full tablet report...
I20260811 00:13:12.179818 18747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:12.183985 18747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:12.184249 18747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:12.202847 18747 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.61:37589
I20260811 00:13:12.202879 18848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.61:37589 every 8 connection(s)
I20260811 00:13:12.204460 18747 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/data/info.pb
I20260811 00:13:12.207935 18851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:12.214166 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 18747
I20260811 00:13:12.214505 18851 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:13:12 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407192, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.61@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM
Aug 11 00:13:12 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.94.61@KRBTEST.COM
I20260811 00:13:12.236198 18858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.217248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60708 (local address 127.7.94.62:36587)
0811 00:13:12.217748 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:12.217756 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:12.218351 (+ 595us) server_negotiation.cc:407] Connection header received
0811 00:13:12.219511 (+ 1160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:12.219519 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:12.219599 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:12.219727 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:12.221468 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.222626 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.224641 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.225297 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.225445 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.229596 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:12.229606 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:12.229612 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:12.229667 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:12.232611 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.233550 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.233554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.233556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.233684 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.234175 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.234180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.234183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.234630 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:12.234798 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:12.235607 (+ 809us) server_negotiation.cc:299] Negotiation successful
0811 00:13:12.235826 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":264,"threads_started":1}
I20260811 00:13:12.237396 18861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.217249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:60708)
0811 00:13:12.218180 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:12.218232 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:12.219266 (+ 1034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:12.219801 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:12.219807 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:12.220704 (+ 897us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:12.221294 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:12.221319 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.222767 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.222771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:12.224495 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:12.224511 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.225518 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.225522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:12.225713 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.226536 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:12.226569 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:12.229393 (+ 2824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:12.232762 (+ 3369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:12.232777 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:12.232790 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:12.233426 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:12.233801 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:12.233812 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:12.233817 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:12.234060 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:12.234782 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:12.234789 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:12.235179 (+ 390us) client_negotiation.cc:769] Sending connection context
0811 00:13:12.235508 (+ 329us) client_negotiation.cc:240] Negotiation successful
0811 00:13:12.235996 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":24,"thread_start_us":537,"threads_started":1}
I20260811 00:13:12.237422 18857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.216397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59000 (local address 127.7.94.61:37589)
0811 00:13:12.217195 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:12.217205 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:12.217287 (+ 82us) server_negotiation.cc:407] Connection header received
0811 00:13:12.219253 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:12.219274 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:12.219555 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:12.219761 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:12.221617 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.223115 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.224791 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.225692 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.225827 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.230680 (+ 4853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:12.230697 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:12.230708 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:12.230735 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:12.233996 (+ 3261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.234775 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.234782 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.234785 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.234933 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.235306 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.235316 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.235319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.235701 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:12.235966 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:12.236319 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 00:13:12.236557 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":568,"thread_start_us":354,"threads_started":1}
I20260811 00:13:12.239470 18851 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:13:12 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407192, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.61@KRBTEST.COM for oryx/127.7.94.61@KRBTEST.COM
I20260811 00:13:12.251385 18861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.240110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:37589 (local address 127.0.0.1:59016)
0811 00:13:12.240309 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:12.240328 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:12.240426 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:12.240721 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:12.240724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:12.241015 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:12.241138 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:12.241145 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.242074 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.242078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:12.242935 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:12.242946 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.243550 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.243553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:12.243681 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.244155 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:12.244171 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:12.246189 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:12.248987 (+ 2798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:12.248995 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:12.248998 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:12.249434 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:12.249804 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:12.249809 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:12.249813 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:12.249935 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:12.250424 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:12.250429 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:12.250579 (+ 150us) client_negotiation.cc:769] Sending connection context
0811 00:13:12.250710 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 00:13:12.251010 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 00:13:12.251435 18857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.240301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59016 (local address 127.7.94.61:37589)
0811 00:13:12.240475 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:12.240479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:12.240493 (+ 14us) server_negotiation.cc:407] Connection header received
0811 00:13:12.240528 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:12.240531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:12.240578 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:12.240657 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:12.241302 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.241969 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.243053 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.243426 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.243524 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.246364 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:12.246368 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:12.246371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:12.246396 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:12.248857 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.249546 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.249553 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.249556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.249697 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.250045 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.250050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.250053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.250314 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:12.250460 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:12.250811 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 00:13:12.251131 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":134,"server-negotiator.queue_time_us":58}
I20260811 00:13:12.256129 18851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4: Bootstrap starting.
I20260811 00:13:12.259194 18851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:12.260192 18851 log.cc:824] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4: Log is configured to *not* fsync() on all Append() calls
I20260811 00:13:12.262773 18851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4: No bootstrap required, opened a new log
I20260811 00:13:12.269761 18851 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } }
I20260811 00:13:12.270087 18851 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:12.270211 18851 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2fdaa9fb639489a89d8ff8c2bb0dfb4, State: Initialized, Role: FOLLOWER
I20260811 00:13:12.270807 18851 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } }
I20260811 00:13:12.271598 18863 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } } }
I20260811 00:13:12.271885 18863 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:13:12.272572 18851 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 00:13:12.281275 18886 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:13:12.281418 18886 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:13:12.284087 18747 master_runner.cc:428] Detected that this master f2fdaa9fb639489a89d8ff8c2bb0dfb4 is joining an existing cluster
I20260811 00:13:12.284202 18747 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.94.61:37589
I20260811 00:13:12.305852 18858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.288181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60722 (local address 127.7.94.62:36587)
0811 00:13:12.288408 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:12.288418 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:12.289061 (+ 643us) server_negotiation.cc:407] Connection header received
0811 00:13:12.289359 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:12.289365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:12.289458 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:12.289534 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:12.291170 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.292282 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.294520 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.295145 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.295254 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.298198 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:12.298208 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:12.298212 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:12.298251 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:12.301708 (+ 3457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.303482 (+ 1774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.303487 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.303491 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.303641 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.304092 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.304098 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.304101 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.304458 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:12.304591 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:12.305279 (+ 688us) server_negotiation.cc:299] Negotiation successful
0811 00:13:12.305506 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 00:13:12.306173 18900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.289020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:37589 (local address 127.0.0.1:59030)
0811 00:13:12.289760 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:12.289774 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:12.289911 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:12.290268 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:12.290271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:12.290951 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:12.291332 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:12.291344 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.292793 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.292802 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:12.294594 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:12.294620 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.295955 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.295963 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:12.296176 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.297145 (+ 969us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:12.297173 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:12.298424 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:12.303223 (+ 4799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:12.303230 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:12.303236 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:12.303789 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:12.304196 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:12.304203 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:12.304209 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:12.304380 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:12.305083 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:12.305089 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:12.305268 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 00:13:12.305429 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 00:13:12.305723 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"mutex_wait_us":24,"thread_start_us":465,"threads_started":1}
I20260811 00:13:12.306721 18857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.289057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59030 (local address 127.7.94.61:37589)
0811 00:13:12.289188 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:12.289192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:12.289854 (+ 662us) server_negotiation.cc:407] Connection header received
0811 00:13:12.290051 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:12.290056 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:12.290137 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:12.290255 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:12.291468 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.292509 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.294747 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.295582 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.295710 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.298635 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:12.298644 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:12.298650 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:12.298689 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:12.303087 (+ 4398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.303928 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.303933 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.303939 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.304073 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.304507 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.304512 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.304520 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.304943 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:12.305117 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:12.305513 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 00:13:12.305854 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":202,"server-negotiator.queue_time_us":41,"spinlock_wait_cycles":15232}
I20260811 00:13:12.306890 18899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.288175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:60722)
0811 00:13:12.288959 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:12.288980 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:12.289178 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:12.289764 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:12.289769 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:12.290554 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:12.290971 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:12.290986 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.292461 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.292466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:12.294352 (+ 1886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:12.294370 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.295572 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.295577 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:12.296007 (+ 430us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.296689 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:12.296722 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:12.297968 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:12.301858 (+ 3890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:12.301863 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:12.301867 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:12.303346 (+ 1479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:12.303792 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:12.303796 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:12.303800 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:12.303960 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:12.304695 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:12.304700 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:12.304954 (+ 254us) client_negotiation.cc:769] Sending connection context
0811 00:13:12.306094 (+ 1140us) client_negotiation.cc:240] Negotiation successful
0811 00:13:12.306404 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"mutex_wait_us":69,"thread_start_us":414,"threads_started":1}
Aug 11 00:13:12 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407189, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.62@KRBTEST.COM for oryx/127.7.94.61@KRBTEST.COM
I20260811 00:13:12.328812 18857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.313290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59046 (local address 127.7.94.61:37589)
0811 00:13:12.313480 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:12.313488 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:12.314138 (+ 650us) server_negotiation.cc:407] Connection header received
0811 00:13:12.314414 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:12.314418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:12.314480 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:12.314558 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:12.315969 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.317819 (+ 1850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.319161 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.319608 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.319751 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.323121 (+ 3370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:12.323126 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:12.323129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:12.323155 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:12.326147 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.326822 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.326827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.326830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.326965 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.327354 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.327360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.327363 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.327618 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:12.327782 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:12.328250 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 00:13:12.328451 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 00:13:12.328867 18903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.313281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:37589 (local address 127.0.0.1:59046)
0811 00:13:12.314004 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:12.314045 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:12.314214 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:12.314640 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:12.314649 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:12.315515 (+ 866us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:12.315835 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:12.315850 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.317959 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.317964 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:12.318964 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:12.318975 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.319775 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.319781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:12.319965 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.320550 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:12.320568 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:12.322934 (+ 2366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:12.326289 (+ 3355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:12.326300 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:12.326308 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:12.326695 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:12.327079 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:12.327085 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:12.327088 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:12.327242 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:12.327770 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:12.327777 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:12.327941 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 00:13:12.328132 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 00:13:12.328452 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":298,"threads_started":1}
WARNING: no policy specified for oryx/127.7.94.60@KRBTEST.COM; defaulting to no policy
I20260811 00:13:12.331130 17914 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.94.61:37589: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4"
member_type: NON_VOTER
last_known_addr {
host: "127.7.94.61"
port: 37589
}
attrs {
promote: true
}
}
dest_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b"
cas_config_opid_index: -1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.94.60@KRBTEST.COM" created.
I20260811 00:13:12.332376 17914 consensus_queue.cc:237] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } }
I20260811 00:13:12.345510 18806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:12.345208 18907 raft_consensus.cc:2932] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f2fdaa9fb639489a89d8ff8c2bb0dfb4 (127.7.94.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } } }
I20260811 00:13:12.349466 18806 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 1 FOLLOWER]: Refusing update from remote peer aad6ba1a2f694580b8fa3dc892b1d83b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 00:13:12.349651 18907 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.94.61:37589
I20260811 00:13:12.350683 18747 master.cc:537] Master@127.7.94.61:37589 shutting down...
I20260811 00:13:12.354717 18906 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } } }
I20260811 00:13:12.355108 18906 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:12.355858 18907 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 00:13:12.364923 17886 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 00:13:12.366755 17886 consensus_peers.cc:487] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b -> Peer f2fdaa9fb639489a89d8ff8c2bb0dfb4 (127.7.94.61:37589): Couldn't send request to peer f2fdaa9fb639489a89d8ff8c2bb0dfb4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.94.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.60.keytab.
Entry for principal oryx/127.7.94.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.60.keytab.
I20260811 00:13:12.371342 18747 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:13:12.371776 18747 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:12.371984 18747 tablet_replica.cc:330] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4: stopping tablet replica
I20260811 00:13:12.373248 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.60:39421
--webserver_interface=127.7.94.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.60.keytab
--principal=oryx/127.7.94.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:39205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:36587,127.7.94.61:37589,127.7.94.60:39421
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:13:12.386765 18747 master.cc:552] Master@127.7.94.61:37589 shutdown complete.
I20260811 00:13:12.386893 18747 master_runner.cc:305] Clearing existing system tablet
I20260811 00:13:12.387394 18747 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:12.387501 18747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:12.390625 18747 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 00:13:12.392339 18915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:12.392759 18747 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:12.393168 18747 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 00:13:12.393311 18747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/wal
uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4"
format_stamp: "Formatted at 2026-08-11 00:13:12 on dist-test-slave-8hhm"
server_key: "60b618afe60a805501f522a7bf7d1368"
server_key_iv: "9eb1337adbd2cbbd960ac3c8fb0680a7"
server_key_version: "encryptionkey@0"
I20260811 00:13:12.399103 18747 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:12.406359 18747 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 00:13:12.406567 18747 log.cc:1196] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 00:13:12.407019 18747 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4: Deleting consensus metadata
I20260811 00:13:12.407320 18747 master_runner.cc:315] Copying system tablet from 127.7.94.62:36587
I20260811 00:13:12.410151 18747 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4: tablet copy: Beginning tablet copy session from remote peer at address 127.7.94.62:36587
I20260811 00:13:12.426119 18858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.410982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60728 (local address 127.7.94.62:36587)
0811 00:13:12.411160 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:12.411168 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:12.411719 (+ 551us) server_negotiation.cc:407] Connection header received
0811 00:13:12.411998 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:12.412004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:12.412097 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:12.412167 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:12.413522 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.414605 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.416361 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.417012 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.417117 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.419751 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:12.419758 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:12.419762 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:12.419802 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:12.423250 (+ 3448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.424035 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.424041 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.424044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.424190 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.424629 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.424634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.424637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.424967 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:12.425115 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:12.425573 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 00:13:12.425785 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 00:13:12.426201 18926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.410946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:60728)
0811 00:13:12.411582 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:12.411607 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:12.411853 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:12.412441 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:12.412445 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:12.413007 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:12.413348 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:12.413365 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.414824 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.414830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:12.416195 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:12.416210 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.417409 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.417415 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:12.417612 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.418389 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:12.418411 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:12.419515 (+ 1104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:12.423391 (+ 3876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:12.423396 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:12.423401 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:12.423906 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:12.424310 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:12.424315 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:12.424319 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:12.424508 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:12.425095 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:12.425100 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:12.425281 (+ 181us) client_negotiation.cc:769] Sending connection context
0811 00:13:12.425453 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 00:13:12.425768 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":336,"threads_started":1}
I20260811 00:13:12.427552 17955 tablet_copy_service.cc:140] P aad6ba1a2f694580b8fa3dc892b1d83b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f2fdaa9fb639489a89d8ff8c2bb0dfb4 ({username='oryx', principal='oryx/127.7.94.61@KRBTEST.COM'} at 127.0.0.1:60728)
I20260811 00:13:12.427815 17955 tablet_copy_service.cc:161] P aad6ba1a2f694580b8fa3dc892b1d83b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f2fdaa9fb639489a89d8ff8c2bb0dfb4 at {username='oryx', principal='oryx/127.7.94.61@KRBTEST.COM'} at 127.0.0.1:60728: session id = f2fdaa9fb639489a89d8ff8c2bb0dfb4-00000000000000000000000000000000
I20260811 00:13:12.432451 17955 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 00:13:12.434808 18747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 00:13:12.442739 18747 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4: tablet copy: Starting download of 0 data blocks...
I20260811 00:13:12.443065 18747 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4: tablet copy: Starting download of 1 WAL segments...
I20260811 00:13:12.445931 18747 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 00:13:12.450419 17955 tablet_copy_service.cc:342] P aad6ba1a2f694580b8fa3dc892b1d83b: Request end of tablet copy session f2fdaa9fb639489a89d8ff8c2bb0dfb4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.94.61@KRBTEST.COM'} at 127.0.0.1:60728
I20260811 00:13:12.450628 17955 tablet_copy_service.cc:433] P aad6ba1a2f694580b8fa3dc892b1d83b: ending tablet copy session f2fdaa9fb639489a89d8ff8c2bb0dfb4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f2fdaa9fb639489a89d8ff8c2bb0dfb4
I20260811 00:13:12.463671 18747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:12.466347 18927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:12.467727 18928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:12.474820 18932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:12.481312 18747 server_base.cc:1034] running on GCE node
I20260811 00:13:12.481647 18747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:12.482292 18747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:12.483525 18747 hybrid_clock.cc:648] HybridClock initialized: now 1786407192483450 us; error 34 us; skew 500 ppm
I20260811 00:13:12.483739 18747 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:12 dist-test-slave-8hhm krb5kdc[17825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407192, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:12.486009 18747 init.cc:377] Logged in from keytab as oryx/127.7.94.61@KRBTEST.COM (short username oryx)
I20260811 00:13:12.490173 18747 webserver.cc:460] Webserver started at http://127.7.94.61:42955/ using document root <none> and password file <none>
I20260811 00:13:12.490553 18747 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:12.490634 18747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:12.493902 18747 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 00:13:12.495911 18944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:12.496285 18747 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260811 00:13:12.496443 18747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/wal
uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4"
format_stamp: "Formatted at 2026-08-11 00:13:12 on dist-test-slave-8hhm"
server_key: "60b618afe60a805501f522a7bf7d1368"
server_key_iv: "9eb1337adbd2cbbd960ac3c8fb0680a7"
server_key_version: "encryptionkey@0"
I20260811 00:13:12.496599 18747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:12.509534 18747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:12.513653 18747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:12.540262 18747 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.61:37589
I20260811 00:13:12.540295 19029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.61:37589 every 8 connection(s)
I20260811 00:13:12.541694 18747 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-1/data/info.pb
I20260811 00:13:12.547003 19033 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:13:12.552794 19033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4: Bootstrap starting.
I20260811 00:13:12.570561 19033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:13:12.571156 19033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4: Bootstrap complete.
I20260811 00:13:12.572427 19033 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } }
I20260811 00:13:12.572718 19033 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f2fdaa9fb639489a89d8ff8c2bb0dfb4, State: Initialized, Role: LEARNER
I20260811 00:13:12.572988 19033 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } }
I20260811 00:13:12.573772 19037 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } } }
I20260811 00:13:12.574026 19033 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:13:12.574227 19037 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [sys.catalog]: This master's current role is: LEARNER
I20260811 00:13:12.583801 19059 catalog_manager.cc:1259] Loaded cluster ID: 98709dcf80214cd5a07efe330d81c3d5
I20260811 00:13:12.583891 19059 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4: loading cluster ID for follower catalog manager: success
I20260811 00:13:12.587283 19059 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4: acquiring CA information for follower catalog manager: success
I20260811 00:13:12.589850 19059 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 00:13:12.689999 18738 heartbeater.cc:503] Master 127.7.94.62:36587 was elected leader, sending a full tablet report...
W20260811 00:13:12.796006 18913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:12.796279 18913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:12.796420 18913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:12.803937 18913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:13:12.804040 18913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:12.804086 18913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:12.804143 18913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:13:12.804188 18913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:13:12.814177 18913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:39205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:36587,127.7.94.61:37589,127.7.94.60:39421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.60.keytab
--principal=oryx/127.7.94.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.60:39421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.94.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:12.815429 18913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:12.816718 18913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:12.828511 19071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:12.828593 19065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:12.828560 19067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:12.829429 18913 server_base.cc:1034] running on GCE node
I20260811 00:13:12.830193 18913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:12.831553 18913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:12.832757 18913 hybrid_clock.cc:648] HybridClock initialized: now 1786407192832697 us; error 48 us; skew 500 ppm
I20260811 00:13:12.833105 18913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:12 dist-test-slave-8hhm krb5kdc[17825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407192, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:12.836257 18913 init.cc:377] Logged in from keytab as oryx/127.7.94.60@KRBTEST.COM (short username oryx)
I20260811 00:13:12.837740 18913 webserver.cc:460] Webserver started at http://127.7.94.60:37831/ using document root <none> and password file <none>
I20260811 00:13:12.838388 18913 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:12.838491 18913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:12.838752 18913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:12.841753 18913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/data/instance:
uuid: "a124e8d010f24686a8793e2134a7febc"
format_stamp: "Formatted at 2026-08-11 00:13:12 on dist-test-slave-8hhm"
server_key: "afe2526796a2e21ae70fdf7bf22c6f60"
server_key_iv: "4be66440a936665f23ce1e0383ad6d35"
server_key_version: "encryptionkey@0"
I20260811 00:13:12.842589 18913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/wal/instance:
uuid: "a124e8d010f24686a8793e2134a7febc"
format_stamp: "Formatted at 2026-08-11 00:13:12 on dist-test-slave-8hhm"
server_key: "afe2526796a2e21ae70fdf7bf22c6f60"
server_key_iv: "4be66440a936665f23ce1e0383ad6d35"
server_key_version: "encryptionkey@0"
I20260811 00:13:12.847690 18913 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:13:12.851167 19084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:12.852514 18913 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:13:12.852697 18913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/wal
uuid: "a124e8d010f24686a8793e2134a7febc"
format_stamp: "Formatted at 2026-08-11 00:13:12 on dist-test-slave-8hhm"
server_key: "afe2526796a2e21ae70fdf7bf22c6f60"
server_key_iv: "4be66440a936665f23ce1e0383ad6d35"
server_key_version: "encryptionkey@0"
I20260811 00:13:12.852885 18913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:12.899768 18913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:12.904455 18913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:12.904757 18913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:12.924319 18913 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.60:39421
I20260811 00:13:12.924361 19177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.60:39421 every 8 connection(s)
I20260811 00:13:12.925904 18913 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/data/info.pb
I20260811 00:13:12.930341 19178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:12.930904 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 18913
I20260811 00:13:12.943375 19178 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:13:12.951913 19187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.941479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:37589 (local address 127.0.0.1:59048)
0811 00:13:12.944281 (+ 2802us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:12.944300 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:12.944460 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:12.944905 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:12.944912 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:12.944943 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:12.945141 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:12.945150 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.947099 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.947104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:12.949096 (+ 1992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:12.949113 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.950882 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.950892 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:12.951158 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.951242 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 00:13:12.951362 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 00:13:12.951523 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2686,"thread_start_us":2589,"threads_started":1}
I20260811 00:13:12.951939 19189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.941948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59048 (local address 127.7.94.61:37589)
0811 00:13:12.942549 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:12.942556 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:12.944400 (+ 1844us) server_negotiation.cc:407] Connection header received
0811 00:13:12.944627 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:12.944634 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:12.944728 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:12.944855 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:12.945345 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.946912 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.949343 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.950667 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.950812 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.951021 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:12.951404 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 00:13:12.951561 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":319,"threads_started":1}
I20260811 00:13:12.953714 18987 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
Aug 11 00:13:12 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.94.60@KRBTEST.COM
I20260811 00:13:12.954396 19037 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aad6ba1a2f694580b8fa3dc892b1d83b. Latest consensus state: current_term: 1 leader_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } } }
I20260811 00:13:12.954675 19037 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [sys.catalog]: This master's current role is: LEARNER
Aug 11 00:13:12 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407192, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.60@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM
I20260811 00:13:12.963991 19184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.933336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52022 (local address 127.7.94.60:39421)
0811 00:13:12.934354 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:12.934376 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:12.937812 (+ 3436us) server_negotiation.cc:407] Connection header received
0811 00:13:12.940294 (+ 2482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:12.940324 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:12.940855 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:12.941317 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:12.943935 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.945833 (+ 1898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.947383 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.948329 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.948501 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.955920 (+ 7419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:12.955948 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:12.955972 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:12.956023 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:12.959633 (+ 3610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.960492 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.960499 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.960504 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.960692 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.961199 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.961202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.961205 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.961870 (+ 665us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:12.962300 (+ 430us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:12.962689 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 00:13:12.963055 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":410,"threads_started":1}
I20260811 00:13:12.964931 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.947557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60730 (local address 127.7.94.62:36587)
0811 00:13:12.948022 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:12.948028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:12.948046 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:13:12.948091 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:12.948095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:12.948172 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:12.948283 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:12.950308 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.951426 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.953114 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.953694 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.953822 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.957832 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:12.957838 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:12.957841 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:12.957898 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:12.961711 (+ 3813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.962575 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.962580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.962584 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.962721 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.963145 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.963150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.963155 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.963395 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:12.963545 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:12.964138 (+ 593us) server_negotiation.cc:299] Negotiation successful
0811 00:13:12.964329 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":233,"threads_started":1}
I20260811 00:13:12.964982 19194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.946212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:60730)
0811 00:13:12.947187 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:12.947238 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:12.947383 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:12.948374 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:12.948382 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:12.949572 (+ 1190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:12.950125 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:12.950140 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.951590 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.951595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:12.952963 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:12.952976 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.953923 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.953931 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:12.954147 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.954857 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:12.954884 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:12.957576 (+ 2692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:12.961850 (+ 4274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:12.961861 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:12.961868 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:12.962441 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:12.962836 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:12.962841 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:12.962844 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:12.963011 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:12.963527 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:12.963534 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:12.963695 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 00:13:12.963997 (+ 302us) client_negotiation.cc:240] Negotiation successful
0811 00:13:12.964328 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":585,"threads_started":1}
I20260811 00:13:12.966308 19178 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:13:12 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407192, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.60@KRBTEST.COM for oryx/127.7.94.61@KRBTEST.COM
I20260811 00:13:12.980027 19194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.966921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:37589 (local address 127.0.0.1:59062)
0811 00:13:12.967089 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:12.967113 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:12.967190 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:12.967576 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:12.967579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:12.967879 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:12.967997 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:12.968004 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.969284 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.969288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:12.970317 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:12.970329 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.971032 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.971044 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:12.971297 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.971808 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:12.971823 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:12.974104 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:12.977838 (+ 3734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:12.977842 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:12.977844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:12.978255 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:12.978625 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:12.978628 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:12.978630 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:12.978817 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:12.979399 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:12.979402 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:12.979510 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 00:13:12.979616 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 00:13:12.979809 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 00:13:12.980202 19189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.967172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59062 (local address 127.7.94.61:37589)
0811 00:13:12.967340 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:12.967344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:12.967358 (+ 14us) server_negotiation.cc:407] Connection header received
0811 00:13:12.967399 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:12.967402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:12.967455 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:12.967543 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:12.968146 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.969120 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.970438 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.970854 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.970941 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.974316 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:12.974321 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:12.974324 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:12.974372 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:12.977728 (+ 3356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.978362 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.978365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.978367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.978532 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.978907 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.978910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.978912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.979301 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:12.979410 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:12.979700 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 00:13:12.979914 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 00:13:12.981055 19178 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:13:12 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407192, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.60@KRBTEST.COM for oryx/127.7.94.60@KRBTEST.COM
I20260811 00:13:12.993088 19194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.981767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:39421 (local address 127.0.0.1:52036)
0811 00:13:12.981898 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:12.981912 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:12.982021 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:12.982342 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:12.982346 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:12.982743 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:12.982931 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:12.982940 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.983868 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.983875 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:12.984673 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:12.984702 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.985438 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.985443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:12.985633 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.986199 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:12.986215 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:12.988081 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:12.990788 (+ 2707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:12.990795 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:12.990799 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:12.991144 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:12.991486 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:12.991492 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:12.991495 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:12.991645 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:12.992158 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:12.992163 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:12.992292 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 00:13:12.992429 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 00:13:12.992678 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 00:13:12.993124 19184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:12.981941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52036 (local address 127.7.94.60:39421)
0811 00:13:12.982086 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:12.982092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:12.982125 (+ 33us) server_negotiation.cc:407] Connection header received
0811 00:13:12.982166 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:12.982170 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:12.982230 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:12.982325 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:12.983067 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.983747 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.984825 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:12.985292 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:12.985420 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:12.988257 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:12.988263 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:12.988267 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:12.988303 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:12.990664 (+ 2361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.991262 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.991267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.991270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.991383 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:12.991777 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:12.991782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:12.991785 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:12.992045 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:12.992193 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:12.992502 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 00:13:12.992814 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":118,"server-negotiator.queue_time_us":47}
I20260811 00:13:12.998178 19178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: Bootstrap starting.
I20260811 00:13:13.001307 19178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:13.002319 19178 log.cc:824] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: Log is configured to *not* fsync() on all Append() calls
I20260811 00:13:13.004732 19178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: No bootstrap required, opened a new log
I20260811 00:13:13.011933 19178 raft_consensus.cc:348] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } }
I20260811 00:13:13.012241 19178 raft_consensus.cc:374] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:13.012326 19178 raft_consensus.cc:729] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a124e8d010f24686a8793e2134a7febc, State: Initialized, Role: FOLLOWER
I20260811 00:13:13.012796 19178 consensus_queue.cc:260] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } }
I20260811 00:13:13.013608 19199 sys_catalog.cc:455] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } } }
I20260811 00:13:13.013952 19199 sys_catalog.cc:458] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:13:13.014603 19178 sys_catalog.cc:564] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [sys.catalog]: configured and running, proceeding with master startup.
W20260811 00:13:13.024865 19217 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:13:13.025000 19217 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:13:13.029043 18913 master_runner.cc:428] Detected that this master a124e8d010f24686a8793e2134a7febc is joining an existing cluster
I20260811 00:13:13.029197 18913 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.94.60:39421
I20260811 00:13:13.054411 19184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:13.033450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52050 (local address 127.7.94.60:39421)
0811 00:13:13.033958 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:13.033967 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:13.034333 (+ 366us) server_negotiation.cc:407] Connection header received
0811 00:13:13.034596 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:13.034602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:13.034686 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:13.034765 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:13.036621 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.038130 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.040906 (+ 2776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.041899 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.042016 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:13.045293 (+ 3277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:13.045304 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:13.045311 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:13.045354 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:13.050345 (+ 4991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:13.051289 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:13.051293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:13.051297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:13.051473 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:13.051955 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:13.051962 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:13.051968 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:13.052476 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:13.052663 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:13.053531 (+ 868us) server_negotiation.cc:299] Negotiation successful
0811 00:13:13.053825 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349}
I20260811 00:13:13.055951 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:13.033555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:39421 (local address 127.0.0.1:52050)
0811 00:13:13.034227 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:13.034254 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:13.034463 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:13.035018 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:13.035025 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:13.035946 (+ 921us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:13.036441 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:13.036458 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.038410 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.038418 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:13.040718 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:13.040739 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.042378 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.042387 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:13.042622 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:13.043506 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:13.043536 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:13.045057 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:13.050550 (+ 5493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:13.050558 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:13.050564 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:13.051163 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:13.051605 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:13.051609 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:13.051613 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:13.051828 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:13.052766 (+ 938us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:13.052773 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:13.053162 (+ 389us) client_negotiation.cc:769] Sending connection context
0811 00:13:13.054687 (+ 1525us) client_negotiation.cc:240] Negotiation successful
0811 00:13:13.055056 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":74,"thread_start_us":433,"threads_started":1}
I20260811 00:13:13.066143 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:13.033155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60736 (local address 127.7.94.62:36587)
0811 00:13:13.033368 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:13.033376 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:13.051363 (+ 17987us) server_negotiation.cc:407] Connection header received
0811 00:13:13.051618 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:13.051624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:13.051732 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:13.051826 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:13.053399 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.054491 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.056659 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.057412 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.057518 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:13.059758 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:13.059763 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:13.059766 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:13.059800 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:13.062973 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:13.063709 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:13.063717 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:13.063721 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:13.063887 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:13.064343 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:13.064349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:13.064353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:13.064730 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:13.064884 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:13.065513 (+ 629us) server_negotiation.cc:299] Negotiation successful
0811 00:13:13.065756 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 00:13:13.066170 19226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:13.034527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:60736)
0811 00:13:13.051255 (+ 16728us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:13.051275 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:13.051472 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:13.052049 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:13.052054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:13.052763 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:13.053194 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:13.053207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.054695 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.054700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:13.056492 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:13.056514 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.057586 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.057598 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:13.057794 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:13.058491 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:13.058514 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:13.059510 (+ 996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:13.063146 (+ 3636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:13.063151 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:13.063156 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:13.063595 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:13.064001 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:13.064012 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:13.064015 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:13.064205 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:13.064872 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:13.064877 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:13.065171 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 00:13:13.065377 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 00:13:13.065690 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16555,"mutex_wait_us":3,"spinlock_wait_cycles":28544,"thread_start_us":16364,"threads_started":1}
I20260811 00:13:13.069991 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 18029
I20260811 00:13:13.073699 19189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:13.051449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59070 (local address 127.7.94.61:37589)
0811 00:13:13.051658 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:13.051665 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:13.056559 (+ 4894us) server_negotiation.cc:407] Connection header received
0811 00:13:13.056663 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:13.056669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:13.056753 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:13.056849 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:13.058940 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.060388 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.062522 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.063197 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.063329 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:13.065346 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:13.065354 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:13.065358 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:13.065401 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:13.069127 (+ 3726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:13.070125 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:13.070131 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:13.070135 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:13.070305 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:13.070768 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:13.070775 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:13.070778 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:13.071163 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:13.071361 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:13.072502 (+ 1141us) server_negotiation.cc:299] Negotiation successful
0811 00:13:13.073315 (+ 813us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 00:13:13.074929 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:13.051827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:37589 (local address 127.0.0.1:59070)
0811 00:13:13.056210 (+ 4383us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:13.056228 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:13.056385 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:13.057375 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:13.057380 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:13.058520 (+ 1140us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:13.058740 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:13.058752 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.060669 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.060675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:13.062352 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:13.062367 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.063365 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.063372 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:13.063541 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:13.064120 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:13.064139 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:13.065132 (+ 993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:13.069325 (+ 4193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:13.069330 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:13.069333 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:13.069976 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:13.070440 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:13.070448 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:13.070452 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:13.070625 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:13.071456 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:13.071460 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:13.071661 (+ 201us) client_negotiation.cc:769] Sending connection context
0811 00:13:13.071838 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 00:13:13.074482 (+ 2644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4278,"thread_start_us":5429,"threads_started":1}
Aug 11 00:13:13 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407189, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.62@KRBTEST.COM for oryx/127.7.94.60@KRBTEST.COM
I20260811 00:13:13.087814 19184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:13.072121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52060 (local address 127.7.94.60:39421)
0811 00:13:13.072276 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:13.072282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:13.072427 (+ 145us) server_negotiation.cc:407] Connection header received
0811 00:13:13.072643 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:13.072649 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:13.072735 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:13.072842 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:13.073732 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.074953 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.076788 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.077414 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.077611 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:13.081244 (+ 3633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:13.081251 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:13.081256 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:13.081294 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:13.084404 (+ 3110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:13.085383 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:13.085389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:13.085391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:13.085542 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:13.085977 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:13.085983 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:13.085986 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:13.086370 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:13.086559 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:13.087141 (+ 582us) server_negotiation.cc:299] Negotiation successful
0811 00:13:13.087394 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 00:13:13.087896 19187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:13.072092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:39421 (local address 127.0.0.1:52060)
0811 00:13:13.072355 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:13.072373 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:13.072489 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:13.072871 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:13.072879 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:13.073374 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:13.073555 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:13.073566 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.075184 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.075189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:13.076612 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:13.076627 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.077848 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.077856 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:13.078035 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:13.078652 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:13.078673 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:13.080999 (+ 2326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:13.084571 (+ 3572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:13.084577 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:13.084581 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:13.085150 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:13.085677 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:13.085682 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:13.085688 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:13.085862 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:13.086547 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:13.086552 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:13.086804 (+ 252us) client_negotiation.cc:769] Sending connection context
0811 00:13:13.087015 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 00:13:13.087341 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260811 00:13:13.089924 17914 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.94.60:39421: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a124e8d010f24686a8793e2134a7febc"
member_type: NON_VOTER
last_known_addr {
host: "127.7.94.60"
port: 39421
}
attrs {
promote: true
}
}
dest_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b"
cas_config_opid_index: 5
Aug 11 00:13:13 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.94.1@KRBTEST.COM
I20260811 00:13:13.091059 17914 consensus_queue.cc:237] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true } }
I20260811 00:13:13.092993 19182 raft_consensus.cc:2932] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a124e8d010f24686a8793e2134a7febc (127.7.94.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true } } }
I20260811 00:13:13.093767 18987 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 1 LEARNER]: Refusing update from remote peer aad6ba1a2f694580b8fa3dc892b1d83b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 00:13:13.094203 19136 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:13.097282 19136 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 1 FOLLOWER]: Refusing update from remote peer aad6ba1a2f694580b8fa3dc892b1d83b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 00:13:13.099071 19229 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 00:13:13.106163 19228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:13.071330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55930 (local address 127.7.94.1:36059)
0811 00:13:13.078007 (+ 6677us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:13.078017 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:13.078046 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:13:13.078116 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:13.078125 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:13.078476 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:13.078676 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:13.080508 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.082047 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.084418 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.085423 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.085555 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:13.091729 (+ 6174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:13.091762 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:13.091780 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:13.091825 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:13.100985 (+ 9160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:13.102219 (+ 1234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:13.102233 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:13.102239 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:13.102440 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:13.102987 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:13.102995 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:13.103000 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:13.103549 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:13.103938 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:13.105457 (+ 1519us) server_negotiation.cc:299] Negotiation successful
0811 00:13:13.105811 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6485,"thread_start_us":6249,"threads_started":1}
I20260811 00:13:13.106488 19229 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:13:13.107764 18262 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:13.108608 18987 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a124e8d010f24686a8793e2134a7febc (127.7.94.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true } } }
I20260811 00:13:13.114442 19037 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true } } }
I20260811 00:13:13.114763 19037 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [sys.catalog]: This master's current role is: LEARNER
I20260811 00:13:13.115638 19182 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.94.60:39421
I20260811 00:13:13.116355 18913 master.cc:537] Master@127.7.94.60:39421 shutting down...
I20260811 00:13:13.116990 19236 raft_consensus.cc:1050] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b: attempting to promote NON_VOTER f2fdaa9fb639489a89d8ff8c2bb0dfb4 to VOTER
I20260811 00:13:13.117122 19237 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true } } }
I20260811 00:13:13.117532 19237 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:13.118013 19236 consensus_queue.cc:237] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true } }
I20260811 00:13:13.126533 19182 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true } } } pending_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true } } }
I20260811 00:13:13.126894 19182 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:13.129712 18987 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 1 LEARNER]: Refusing update from remote peer aad6ba1a2f694580b8fa3dc892b1d83b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 00:13:13.130535 19236 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 00:13:13.133638 19182 raft_consensus.cc:2932] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f2fdaa9fb639489a89d8ff8c2bb0dfb4 (127.7.94.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true } } }
I20260811 00:13:13.134411 18986 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f2fdaa9fb639489a89d8ff8c2bb0dfb4 (127.7.94.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true } } }
W20260811 00:13:13.137885 17884 consensus_peers.cc:487] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b -> Peer a124e8d010f24686a8793e2134a7febc (127.7.94.60:39421): Couldn't send request to peer a124e8d010f24686a8793e2134a7febc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 00:13:13.138404 19037 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true } } }
I20260811 00:13:13.138674 19037 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:13:13.139586 19237 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true } } }
I20260811 00:13:13.139628 19229 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true } } }
I20260811 00:13:13.139919 19229 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:13.139919 19237 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:13.146994 19136 raft_consensus.cc:858] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true } }
W20260811 00:13:13.147176 19136 raft_consensus.cc:1565] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true } }
W20260811 00:13:13.169550 19092 tablet.cc:2369] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:13:13.179132 19136 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f2fdaa9fb639489a89d8ff8c2bb0dfb4 (127.7.94.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } } }
I20260811 00:13:13.184998 19199 sys_catalog.cc:455] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [sys.catalog]: SysCatalogTable state changed. Reason: New leader aad6ba1a2f694580b8fa3dc892b1d83b. Latest consensus state: current_term: 1 leader_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } } }
I20260811 00:13:13.185019 19248 sys_catalog.cc:455] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } } }
I20260811 00:13:13.185329 19199 sys_catalog.cc:458] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 00:13:13.185412 19248 sys_catalog.cc:458] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 00:13:13.184310 19254 sys_catalog.cc:455] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: NON_VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: true } } }
I20260811 00:13:13.186244 19254 sys_catalog.cc:458] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 00:13:13.188350 18913 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260811 00:13:13.188889 18913 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260811 00:13:13.189064 18913 tablet_replica.cc:330] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: stopping tablet replica
I20260811 00:13:13.196700 18913 master.cc:552] Master@127.7.94.60:39421 shutdown complete.
I20260811 00:13:13.196838 18913 master_runner.cc:305] Clearing existing system tablet
I20260811 00:13:13.197273 18913 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:13.197389 18913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:13.205932 18913 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.000s sys 0.008s
I20260811 00:13:13.207679 19256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:13.208040 18913 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:13.208292 18913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260811 00:13:13.208405 18913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/wal
uuid: "a124e8d010f24686a8793e2134a7febc"
format_stamp: "Formatted at 2026-08-11 00:13:12 on dist-test-slave-8hhm"
server_key: "afe2526796a2e21ae70fdf7bf22c6f60"
server_key_iv: "4be66440a936665f23ce1e0383ad6d35"
server_key_version: "encryptionkey@0"
I20260811 00:13:13.213099 18913 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:13.220594 18913 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 00:13:13.220774 18913 log.cc:1196] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 00:13:13.221277 18913 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: Deleting consensus metadata
I20260811 00:13:13.221540 18913 master_runner.cc:315] Copying system tablet from 127.7.94.62:36587
I20260811 00:13:13.227298 18913 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: tablet copy: Beginning tablet copy session from remote peer at address 127.7.94.62:36587
I20260811 00:13:13.234771 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 18029
I20260811 00:13:13.250056 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:13.228183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60750 (local address 127.7.94.62:36587)
0811 00:13:13.228413 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:13.228420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:13.233684 (+ 5264us) server_negotiation.cc:407] Connection header received
0811 00:13:13.233905 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:13.233914 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:13.234031 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:13.234145 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:13.235305 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.236550 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.238985 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.239693 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.239837 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:13.243090 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:13.243098 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:13.243103 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:13.243142 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:13.246757 (+ 3615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:13.247591 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:13.247599 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:13.247603 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:13.247766 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:13.248246 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:13.248251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:13.248254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:13.248604 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:13.248769 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:13.249404 (+ 635us) server_negotiation.cc:299] Negotiation successful
0811 00:13:13.249654 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260811 00:13:13.250227 19271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:13.228137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:60750)
0811 00:13:13.233539 (+ 5402us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:13.233555 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:13.233761 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:13.234308 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:13.234312 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:13.234812 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:13.235118 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:13.235134 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.236767 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.236775 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:13.238826 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:13.238845 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.240091 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.240097 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:13.240270 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:13.241566 (+ 1296us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:13.241602 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:13.242882 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:13.246900 (+ 4018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:13.246908 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:13.246914 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:13.247442 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:13.247896 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:13.247904 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:13.247911 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:13.248115 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:13.248755 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:13.248763 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:13.249024 (+ 261us) client_negotiation.cc:769] Sending connection context
0811 00:13:13.249244 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 00:13:13.249633 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5267,"thread_start_us":5153,"threads_started":1}
I20260811 00:13:13.251065 17955 tablet_copy_service.cc:140] P aad6ba1a2f694580b8fa3dc892b1d83b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a124e8d010f24686a8793e2134a7febc ({username='oryx', principal='oryx/127.7.94.60@KRBTEST.COM'} at 127.0.0.1:60750)
I20260811 00:13:13.251231 17955 tablet_copy_service.cc:161] P aad6ba1a2f694580b8fa3dc892b1d83b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a124e8d010f24686a8793e2134a7febc at {username='oryx', principal='oryx/127.7.94.60@KRBTEST.COM'} at 127.0.0.1:60750: session id = a124e8d010f24686a8793e2134a7febc-00000000000000000000000000000000
I20260811 00:13:13.254913 17955 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 00:13:13.256795 18913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 00:13:13.262765 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:36059
--local_ip_for_outbound_sockets=127.7.94.1
--tserver_master_addrs=127.7.94.62:36587
--webserver_port=34911
--webserver_interface=127.7.94.1
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.1.keytab
--principal=oryx/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:39205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.94.62:36587,127.7.94.61:37589
--tserver_master_addrs=127.7.94.62:36587,127.7.94.61:37589,127.7.94.60:39421 with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:13:13.263533 18913 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: tablet copy: Starting download of 0 data blocks...
I20260811 00:13:13.263849 18913 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: tablet copy: Starting download of 1 WAL segments...
I20260811 00:13:13.266075 18913 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 00:13:13.269562 17955 tablet_copy_service.cc:342] P aad6ba1a2f694580b8fa3dc892b1d83b: Request end of tablet copy session a124e8d010f24686a8793e2134a7febc-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.94.60@KRBTEST.COM'} at 127.0.0.1:60750
I20260811 00:13:13.269711 17955 tablet_copy_service.cc:433] P aad6ba1a2f694580b8fa3dc892b1d83b: ending tablet copy session a124e8d010f24686a8793e2134a7febc-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a124e8d010f24686a8793e2134a7febc
I20260811 00:13:13.283209 18913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:13.286384 19276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:13.287741 19277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:13.294587 19279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:13.295130 18913 server_base.cc:1034] running on GCE node
I20260811 00:13:13.295526 18913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:13.296288 18913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:13.297482 18913 hybrid_clock.cc:648] HybridClock initialized: now 1786407193297428 us; error 31 us; skew 500 ppm
I20260811 00:13:13.297792 18913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:13 dist-test-slave-8hhm krb5kdc[17825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407193, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:13.300118 18913 init.cc:377] Logged in from keytab as oryx/127.7.94.60@KRBTEST.COM (short username oryx)
I20260811 00:13:13.306308 18913 webserver.cc:460] Webserver started at http://127.7.94.60:42017/ using document root <none> and password file <none>
I20260811 00:13:13.306658 18913 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:13.306759 18913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:13.309966 18913 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 00:13:13.311968 19290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:13.312494 18913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260811 00:13:13.312630 18913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/wal
uuid: "a124e8d010f24686a8793e2134a7febc"
format_stamp: "Formatted at 2026-08-11 00:13:12 on dist-test-slave-8hhm"
server_key: "afe2526796a2e21ae70fdf7bf22c6f60"
server_key_iv: "4be66440a936665f23ce1e0383ad6d35"
server_key_version: "encryptionkey@0"
I20260811 00:13:13.312826 18913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:13.324146 18913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:13.328086 18913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:13.363801 18913 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.60:39421
I20260811 00:13:13.363940 19391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.60:39421 every 8 connection(s)
I20260811 00:13:13.364799 18913 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/master-2/data/info.pb
I20260811 00:13:13.370010 19393 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:13:13.374114 19393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: Bootstrap starting.
I20260811 00:13:13.389902 19393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:13:13.390482 19393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: Bootstrap complete.
I20260811 00:13:13.391579 19393 raft_consensus.cc:348] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true } }
I20260811 00:13:13.391911 19393 raft_consensus.cc:729] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a124e8d010f24686a8793e2134a7febc, State: Initialized, Role: LEARNER
I20260811 00:13:13.392167 19393 consensus_queue.cc:260] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true } }
I20260811 00:13:13.393173 19393 sys_catalog.cc:564] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:13:13.392947 19396 sys_catalog.cc:455] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true } } }
I20260811 00:13:13.393437 19396 sys_catalog.cc:458] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [sys.catalog]: This master's current role is: LEARNER
I20260811 00:13:13.405386 19416 catalog_manager.cc:1259] Loaded cluster ID: 98709dcf80214cd5a07efe330d81c3d5
I20260811 00:13:13.405532 19416 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: loading cluster ID for follower catalog manager: success
I20260811 00:13:13.408008 19416 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: acquiring CA information for follower catalog manager: success
I20260811 00:13:13.409940 19416 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 00:13:13.658520 19272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:13.658865 19272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:13.659026 19272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:13.667083 19272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:13.667243 19272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:13.667407 19272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:13:13.678483 19272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:39205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.1.keytab
--principal=oryx/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:36059
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=34911
--tserver_master_addrs=127.7.94.62:36587,127.7.94.61:37589,127.7.94.60:39421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:13.680095 19272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:13.681427 19272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 00:13:13.685323 19424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:13.679253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:39421 (local address 127.0.0.1:52072)
0811 00:13:13.679715 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:13.679745 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:13.679902 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:13.680439 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:13.680445 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:13.680474 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:13.680684 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:13.680694 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.681962 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.681969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:13.683363 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:13.683375 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.684453 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.684460 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:13.684657 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:13.684693 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 00:13:13.684811 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 00:13:13.684966 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":234,"threads_started":1}
I20260811 00:13:13.685359 19428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:13.679420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52072 (local address 127.7.94.60:39421)
0811 00:13:13.680085 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:13.680090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:13.680106 (+ 16us) server_negotiation.cc:407] Connection header received
0811 00:13:13.680168 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:13.680172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:13.680269 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:13.680399 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:13.680829 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.681828 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.683527 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.684285 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.684434 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:13.684681 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:13.684834 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 00:13:13.684965 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":343,"threads_started":1}
I20260811 00:13:13.686182 19334 raft_consensus.cc:1203] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 00:13:13.686620 19396 sys_catalog.cc:455] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [sys.catalog]: SysCatalogTable state changed. Reason: New leader aad6ba1a2f694580b8fa3dc892b1d83b. Latest consensus state: current_term: 1 leader_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true } } }
I20260811 00:13:13.686901 19396 sys_catalog.cc:458] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [sys.catalog]: This master's current role is: LEARNER
W20260811 00:13:13.696043 19433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:13.696079 19436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:13.695971 19434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:13.697614 19272 server_base.cc:1034] running on GCE node
I20260811 00:13:13.698272 19272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:13.699156 19272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:13.700443 19272 hybrid_clock.cc:648] HybridClock initialized: now 1786407193700385 us; error 52 us; skew 500 ppm
I20260811 00:13:13.700835 19272 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:13 dist-test-slave-8hhm krb5kdc[17825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407193, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:13.704017 19272 init.cc:377] Logged in from keytab as oryx/127.7.94.1@KRBTEST.COM (short username oryx)
I20260811 00:13:13.705542 19272 webserver.cc:460] Webserver started at http://127.7.94.1:34911/ using document root <none> and password file <none>
I20260811 00:13:13.706324 19272 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:13.706529 19272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:13.712200 19272 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 00:13:13.715190 19445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:13.716475 19272 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:13:13.716596 19272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "c488dd19f66d45419d4054c024614935"
format_stamp: "Formatted at 2026-08-11 00:13:10 on dist-test-slave-8hhm"
server_key: "ee25d3eb77d13be02b357fc27442f6d1"
server_key_iv: "7e9308363e11f173fe9967d94eb99240"
server_key_version: "encryptionkey@0"
I20260811 00:13:13.717607 19272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:13.746829 19272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:13.751597 19272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:13.751865 19272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:13.752821 19272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:13.754253 19272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:13.754346 19272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:13.754427 19272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:13.754485 19272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:13.800170 19272 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:36059
I20260811 00:13:13.800251 19631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:36059 every 8 connection(s)
I20260811 00:13:13.801735 19272 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:13:13.806149 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 19272
I20260811 00:13:13.806589 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 18282
Aug 11 00:13:13 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407193, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.60@KRBTEST.COM
I20260811 00:13:13.837466 19428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:13.804592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:44107 (local address 127.7.94.60:39421)
0811 00:13:13.804855 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:13.804862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:13.808114 (+ 3252us) server_negotiation.cc:407] Connection header received
0811 00:13:13.811793 (+ 3679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:13.811803 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:13.811915 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:13.812006 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:13.815020 (+ 3014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.816509 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.819958 (+ 3449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.820877 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.821000 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:13.827095 (+ 6095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:13.827105 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:13.827113 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:13.827168 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:13.831146 (+ 3978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:13.833148 (+ 2002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:13.833154 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:13.833157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:13.833333 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:13.833961 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:13.833966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:13.833970 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:13.834442 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:13.834646 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:13.836852 (+ 2206us) server_negotiation.cc:299] Negotiation successful
0811 00:13:13.837125 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 00:13:13.837631 19640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:13.805501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:39421 (local address 127.7.94.1:44107)
0811 00:13:13.807941 (+ 2440us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:13.807994 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:13.811356 (+ 3362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:13.812685 (+ 1329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:13.812701 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:13.813730 (+ 1029us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:13.814767 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:13.814799 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.816796 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.816805 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:13.819725 (+ 2920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:13.819747 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.821439 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.821449 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:13.821886 (+ 437us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:13.822827 (+ 941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:13.822869 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:13.826780 (+ 3911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:13.832220 (+ 5440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:13.832233 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:13.832249 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:13.832995 (+ 746us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:13.833477 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:13.833483 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:13.833486 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:13.833771 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:13.834722 (+ 951us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:13.834730 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:13.835192 (+ 462us) client_negotiation.cc:769] Sending connection context
0811 00:13:13.835545 (+ 353us) client_negotiation.cc:240] Negotiation successful
0811 00:13:13.836093 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2142,"thread_start_us":23591,"threads_started":1}
I20260811 00:13:13.839401 19633 heartbeater.cc:347] Connected to a master server at 127.7.94.60:39421
I20260811 00:13:13.839789 19633 heartbeater.cc:464] Registering TS with master...
Aug 11 00:13:13 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407193, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM
I20260811 00:13:13.840824 19633 heartbeater.cc:511] Master 127.7.94.60:39421 requested a full tablet report, sending...
Aug 11 00:13:13 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407193, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.61@KRBTEST.COM
I20260811 00:13:13.844189 19317 ts_manager.cc:194] Registered new tserver with Master: c488dd19f66d45419d4054c024614935 (127.7.94.1:36059)
Aug 11 00:13:13 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.94.2@KRBTEST.COM
I20260811 00:13:13.850910 19641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:13.805594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.1:57155)
0811 00:13:13.828088 (+ 22494us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:13.828114 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:13.828267 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:13.829721 (+ 1454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:13.829729 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:13.831667 (+ 1938us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:13.832010 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:13.832022 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.833511 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.833515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:13.835274 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:13.835287 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.836449 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.836456 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:13.836662 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:13.838576 (+ 1914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:13.838597 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:13.841632 (+ 3035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:13.846414 (+ 4782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:13.846419 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:13.846424 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:13.847015 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:13.848918 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:13.848922 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:13.848925 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:13.849123 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:13.849930 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:13.849935 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:13.850102 (+ 167us) client_negotiation.cc:769] Sending connection context
0811 00:13:13.850243 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 00:13:13.850466 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22346,"mutex_wait_us":1177,"thread_start_us":1869,"threads_started":1}
I20260811 00:13:13.851835 19638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:13.805402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:57155 (local address 127.7.94.62:36587)
0811 00:13:13.827320 (+ 21918us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:13.827342 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:13.829063 (+ 1721us) server_negotiation.cc:407] Connection header received
0811 00:13:13.829144 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:13.829152 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:13.829537 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:13.829663 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:13.832190 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.833353 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.835443 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.836043 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.836151 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:13.841884 (+ 5733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:13.841891 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:13.841896 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:13.841937 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:13.846265 (+ 4328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:13.847122 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:13.847128 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:13.847131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:13.847323 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:13.849300 (+ 1977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:13.849308 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:13.849314 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:13.849782 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:13.849950 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:13.851087 (+ 1137us) server_negotiation.cc:299] Negotiation successful
0811 00:13:13.851354 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21739,"thread_start_us":21605,"threads_started":1}
I20260811 00:13:13.852348 19650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:13.808872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:52255 (local address 127.7.94.61:37589)
0811 00:13:13.826789 (+ 17917us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:13.826799 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:13.829429 (+ 2630us) server_negotiation.cc:407] Connection header received
0811 00:13:13.830681 (+ 1252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:13.830692 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:13.830828 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:13.830937 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:13.833867 (+ 2930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.835754 (+ 1887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.839163 (+ 3409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.840078 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.840180 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:13.845335 (+ 5155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:13.845345 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:13.845350 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:13.845402 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:13.848736 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:13.849622 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:13.849630 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:13.849634 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:13.849772 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:13.850262 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:13.850268 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:13.850272 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:13.850697 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:13.850866 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:13.851711 (+ 845us) server_negotiation.cc:299] Negotiation successful
0811 00:13:13.851933 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17748,"thread_start_us":17931,"threads_started":1}
I20260811 00:13:13.852844 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:13.808558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:37589 (local address 127.7.94.1:52255)
0811 00:13:13.829304 (+ 20746us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:13.829328 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:13.829567 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:13.831272 (+ 1705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:13.831276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:13.832912 (+ 1636us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:13.833621 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:13.833643 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.836009 (+ 2366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.836017 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:13.838925 (+ 2908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:13.838946 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.840553 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.840562 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:13.840776 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:13.841525 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:13.841551 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:13.845087 (+ 3536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:13.848879 (+ 3792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:13.848886 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:13.848893 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:13.849481 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:13.849930 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:13.849936 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:13.849939 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:13.850116 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:13.850930 (+ 814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:13.850937 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:13.851437 (+ 500us) client_negotiation.cc:769] Sending connection context
0811 00:13:13.851573 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 00:13:13.851864 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20558,"thread_start_us":20211,"threads_started":1}
I20260811 00:13:13.853935 19634 heartbeater.cc:347] Connected to a master server at 127.7.94.62:36587
I20260811 00:13:13.854128 19634 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:13.854636 19634 heartbeater.cc:511] Master 127.7.94.62:36587 requested a full tablet report, sending...
I20260811 00:13:13.854897 19635 heartbeater.cc:347] Connected to a master server at 127.7.94.61:37589
I20260811 00:13:13.855051 19635 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:13.855569 19635 heartbeater.cc:511] Master 127.7.94.61:37589 requested a full tablet report, sending...
I20260811 00:13:13.856106 17914 ts_manager.cc:194] Re-registered known tserver with Master: c488dd19f66d45419d4054c024614935 (127.7.94.1:36059)
I20260811 00:13:13.856839 19645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:13.808189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52608 (local address 127.7.94.2:35337)
0811 00:13:13.830933 (+ 22744us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:13.830944 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:13.831709 (+ 765us) server_negotiation.cc:407] Connection header received
0811 00:13:13.831837 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:13.831847 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:13.832172 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:13.832349 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:13.836552 (+ 4203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.838590 (+ 2038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.840900 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:13.841973 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:13.842148 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:13.848641 (+ 6493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:13.848653 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:13.848667 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:13.848713 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:13.853153 (+ 4440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:13.854135 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:13.854142 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:13.854144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:13.854361 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:13.854872 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:13.854877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:13.854881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:13.855372 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:13.855672 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:13.856106 (+ 434us) server_negotiation.cc:299] Negotiation successful
0811 00:13:13.856390 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22505,"thread_start_us":21790,"threads_started":1}
I20260811 00:13:13.857312 17914 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:57155
I20260811 00:13:13.857417 18969 ts_manager.cc:194] Registered new tserver with Master: c488dd19f66d45419d4054c024614935 (127.7.94.1:36059)
I20260811 00:13:13.859066 18498 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:13.953998 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 18282
I20260811 00:13:13.982396 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:35337
--local_ip_for_outbound_sockets=127.7.94.2
--tserver_master_addrs=127.7.94.62:36587
--webserver_port=39273
--webserver_interface=127.7.94.2
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.2.keytab
--principal=oryx/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:39205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.94.62:36587,127.7.94.61:37589
--tserver_master_addrs=127.7.94.62:36587,127.7.94.61:37589,127.7.94.60:39421 with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:13:14.269055 19423 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: NON_VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: true } } }
I20260811 00:13:14.269523 19423 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:14.271214 19423 raft_consensus.cc:1050] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b: attempting to promote NON_VOTER a124e8d010f24686a8793e2134a7febc to VOTER
I20260811 00:13:14.271816 19423 consensus_queue.cc:237] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false } }
I20260811 00:13:14.275496 19334 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 1 LEARNER]: Refusing update from remote peer aad6ba1a2f694580b8fa3dc892b1d83b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 00:13:14.275830 18986 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 1 FOLLOWER]: Refusing update from remote peer aad6ba1a2f694580b8fa3dc892b1d83b: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 00:13:14.276136 19654 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 00:13:14.276571 19423 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 00:13:14.280033 19653 raft_consensus.cc:2932] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, a124e8d010f24686a8793e2134a7febc (127.7.94.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false } } }
I20260811 00:13:14.280486 18986 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, a124e8d010f24686a8793e2134a7febc (127.7.94.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false } } }
I20260811 00:13:14.285111 19653 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false } } }
I20260811 00:13:14.285537 19653 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:14.285828 19423 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false } } }
I20260811 00:13:14.286125 19423 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:14.287931 19663 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false } } }
I20260811 00:13:14.288265 19663 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:13:14.292408 19653 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false } } }
I20260811 00:13:14.292713 19653 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:14.293128 19334 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, a124e8d010f24686a8793e2134a7febc (127.7.94.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false } } }
I20260811 00:13:14.301584 19664 sys_catalog.cc:455] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false } } }
I20260811 00:13:14.301918 19664 sys_catalog.cc:458] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [sys.catalog]: This master's current role is: FOLLOWER
W20260811 00:13:14.343724 19652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:14.344048 19652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:14.344219 19652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:14.351480 19652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:14.351610 19652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:14.351797 19652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:13:14.362046 19652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:39205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.2.keytab
--principal=oryx/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:35337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=39273
--tserver_master_addrs=127.7.94.62:36587,127.7.94.61:37589,127.7.94.60:39421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:14.363340 19652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:14.364619 19652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:14.378393 19671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:14.378430 19674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:14.378350 19672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:14.379314 19652 server_base.cc:1034] running on GCE node
I20260811 00:13:14.379923 19652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:14.380885 19652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:14.382110 19652 hybrid_clock.cc:648] HybridClock initialized: now 1786407194382059 us; error 59 us; skew 500 ppm
I20260811 00:13:14.382436 19652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:14 dist-test-slave-8hhm krb5kdc[17825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407194, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:14.385722 19652 init.cc:377] Logged in from keytab as oryx/127.7.94.2@KRBTEST.COM (short username oryx)
I20260811 00:13:14.387207 19652 webserver.cc:460] Webserver started at http://127.7.94.2:39273/ using document root <none> and password file <none>
I20260811 00:13:14.387884 19652 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:14.387993 19652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:14.393332 19652 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:13:14.396056 19687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:14.397812 19652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:13:14.397941 19652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "6c6905d72f434b7f86d5d92f6a46184c"
format_stamp: "Formatted at 2026-08-11 00:13:11 on dist-test-slave-8hhm"
server_key: "651738787c92206177f544ada2bfcb3c"
server_key_iv: "8f9958243e6fb031b4d9c6db66bd4e64"
server_key_version: "encryptionkey@0"
I20260811 00:13:14.398888 19652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:14.413980 19652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:14.418480 19652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:14.418706 19652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:14.419646 19652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:14.420962 19652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:14.421049 19652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:14.421134 19652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:14.421193 19652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:14.454681 19652 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:35337
I20260811 00:13:14.454744 19876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:35337 every 8 connection(s)
I20260811 00:13:14.456113 19652 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:13:14.457753 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 19652
I20260811 00:13:14.458119 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 18511
Aug 11 00:13:14 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.94.3@KRBTEST.COM
Aug 11 00:13:14 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407194, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.60@KRBTEST.COM
Aug 11 00:13:14 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407194, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM
Aug 11 00:13:14 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407194, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.61@KRBTEST.COM
I20260811 00:13:14.517433 19893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:14.459474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43080 (local address 127.7.94.3:34285)
0811 00:13:14.491495 (+ 32021us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:14.491508 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:14.491547 (+ 39us) server_negotiation.cc:407] Connection header received
0811 00:13:14.491650 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:14.491660 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:14.492001 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:14.492213 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:14.496245 (+ 4032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:14.497830 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:14.499786 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:14.500625 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:14.500752 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:14.508311 (+ 7559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:14.508340 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:14.508357 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:14.508398 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:14.513508 (+ 5110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:14.514383 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:14.514393 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:14.514399 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:14.514574 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:14.515029 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:14.515040 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:14.515043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:14.515484 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:14.515779 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:14.516661 (+ 882us) server_negotiation.cc:299] Negotiation successful
0811 00:13:14.516932 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31805,"thread_start_us":36783,"threads_started":1}
I20260811 00:13:14.518288 19895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:14.465886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:56035 (local address 127.7.94.62:36587)
0811 00:13:14.481681 (+ 15795us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:14.481690 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:14.490353 (+ 8663us) server_negotiation.cc:407] Connection header received
0811 00:13:14.498761 (+ 8408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:14.498771 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:14.498911 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:14.499035 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:14.500646 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:14.502136 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:14.504636 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:14.505035 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:14.505226 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:14.510054 (+ 4828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:14.510064 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:14.510070 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:14.510125 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:14.514287 (+ 4162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:14.515228 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:14.515233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:14.515239 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:14.515401 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:14.515970 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:14.515975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:14.515979 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:14.516321 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:14.516440 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:14.517633 (+ 1193us) server_negotiation.cc:299] Negotiation successful
0811 00:13:14.517911 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15632,"thread_start_us":15820,"threads_started":1}
I20260811 00:13:14.518373 19889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:14.458949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:43941 (local address 127.7.94.60:39421)
0811 00:13:14.484677 (+ 25728us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:14.484687 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:14.492514 (+ 7827us) server_negotiation.cc:407] Connection header received
0811 00:13:14.494317 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:14.494326 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:14.494437 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:14.494559 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:14.497989 (+ 3430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:14.499282 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:14.502110 (+ 2828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:14.502805 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:14.502942 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:14.509070 (+ 6128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:14.509080 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:14.509085 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:14.509127 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:14.513430 (+ 4303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:14.515509 (+ 2079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:14.515515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:14.515518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:14.515683 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:14.516150 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:14.516154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:14.516157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:14.516669 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:14.516805 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:14.517808 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0811 00:13:14.518019 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25519,"thread_start_us":26557,"threads_started":1}
I20260811 00:13:14.519332 18729 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:14.520509 19896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:14.474710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:39421 (local address 127.7.94.2:43941)
0811 00:13:14.490864 (+ 16154us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:14.490884 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:14.493664 (+ 2780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:14.495128 (+ 1464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:14.495145 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:14.496696 (+ 1551us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:14.497762 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:14.497787 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:14.499478 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:14.499484 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:14.501935 (+ 2451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:14.501950 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:14.503287 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:14.503296 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:14.503628 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:14.504423 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:14.504469 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:14.508782 (+ 4313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:14.513661 (+ 4879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:14.513670 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:14.513688 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:14.515403 (+ 1715us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:14.515829 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:14.515833 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:14.515837 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:14.515988 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:14.516868 (+ 880us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:14.516872 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:14.517039 (+ 167us) client_negotiation.cc:769] Sending connection context
0811 00:13:14.517397 (+ 358us) client_negotiation.cc:240] Negotiation successful
0811 00:13:14.518953 (+ 1556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16032,"mutex_wait_us":1,"spinlock_wait_cycles":20224,"thread_start_us":14727,"threads_started":1}
I20260811 00:13:14.522172 19878 heartbeater.cc:347] Connected to a master server at 127.7.94.60:39421
I20260811 00:13:14.522569 19878 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:14.523437 19878 heartbeater.cc:511] Master 127.7.94.60:39421 requested a full tablet report, sending...
I20260811 00:13:14.525032 19317 ts_manager.cc:194] Registered new tserver with Master: 6c6905d72f434b7f86d5d92f6a46184c (127.7.94.2:35337)
I20260811 00:13:14.525408 19899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:14.491172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:43293 (local address 127.7.94.61:37589)
0811 00:13:14.498122 (+ 6950us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:14.498127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:14.505810 (+ 7683us) server_negotiation.cc:407] Connection header received
0811 00:13:14.506044 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:14.506049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:14.506113 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:14.506231 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:14.507662 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:14.509154 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:14.511321 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:14.512018 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:14.512135 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:14.516629 (+ 4494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:14.516635 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:14.516639 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:14.516685 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:14.520896 (+ 4211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:14.522069 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:14.522077 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:14.522083 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:14.522300 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:14.522848 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:14.522856 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:14.522862 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:14.523312 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:14.523491 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:14.524498 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0811 00:13:14.524827 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6859,"thread_start_us":6670,"threads_started":1}
I20260811 00:13:14.526109 19900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:14.496743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:37589 (local address 127.7.94.2:43293)
0811 00:13:14.505705 (+ 8962us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:14.505726 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:14.505895 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:14.506253 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:14.506257 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:14.506924 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:14.507470 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:14.507485 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:14.509392 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:14.509399 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:14.511138 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:14.511161 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:14.512430 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:14.512436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:14.512621 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:14.513165 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:14.513186 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:14.516430 (+ 3244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:14.521085 (+ 4655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:14.521093 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:14.521113 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:14.521899 (+ 786us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:14.522459 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:14.522467 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:14.522473 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:14.522691 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:14.523567 (+ 876us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:14.523575 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:14.523756 (+ 181us) client_negotiation.cc:769] Sending connection context
0811 00:13:14.523941 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 00:13:14.525671 (+ 1730us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8814,"thread_start_us":7426,"threads_started":1}
I20260811 00:13:14.526400 19898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:14.466475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.2:56035)
0811 00:13:14.490113 (+ 23638us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:14.490179 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:14.498431 (+ 8252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:14.499386 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:14.499394 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:14.500057 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:14.500434 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:14.500448 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:14.502426 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:14.502434 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:14.504489 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:14.504500 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:14.505263 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:14.505269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:14.505490 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:14.506557 (+ 1067us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:14.506588 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:14.509791 (+ 3203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:14.514521 (+ 4730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:14.514527 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:14.514530 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:14.515099 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:14.515549 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:14.515556 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:14.515560 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:14.515838 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:14.516550 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:14.516559 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:14.517052 (+ 493us) client_negotiation.cc:769] Sending connection context
0811 00:13:14.525566 (+ 8514us) client_negotiation.cc:240] Negotiation successful
0811 00:13:14.525853 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23200,"thread_start_us":23910,"threads_started":1}
I20260811 00:13:14.527063 19885 heartbeater.cc:347] Connected to a master server at 127.7.94.61:37589
I20260811 00:13:14.527197 19885 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:14.527297 19879 heartbeater.cc:347] Connected to a master server at 127.7.94.62:36587
I20260811 00:13:14.527388 19879 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:14.528245 19885 heartbeater.cc:511] Master 127.7.94.61:37589 requested a full tablet report, sending...
I20260811 00:13:14.528543 19879 heartbeater.cc:511] Master 127.7.94.62:36587 requested a full tablet report, sending...
I20260811 00:13:14.529625 18969 ts_manager.cc:194] Registered new tserver with Master: 6c6905d72f434b7f86d5d92f6a46184c (127.7.94.2:35337)
I20260811 00:13:14.529966 17914 ts_manager.cc:194] Re-registered known tserver with Master: 6c6905d72f434b7f86d5d92f6a46184c (127.7.94.2:35337)
I20260811 00:13:14.530833 17914 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:56035
I20260811 00:13:14.609335 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 18511
I20260811 00:13:14.637449 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:34285
--local_ip_for_outbound_sockets=127.7.94.3
--tserver_master_addrs=127.7.94.62:36587
--webserver_port=46361
--webserver_interface=127.7.94.3
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.3.keytab
--principal=oryx/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:39205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.94.62:36587,127.7.94.61:37589
--tserver_master_addrs=127.7.94.62:36587,127.7.94.61:37589,127.7.94.60:39421 with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:13:14.860877 19634 heartbeater.cc:503] Master 127.7.94.62:36587 was elected leader, sending a full tablet report...
W20260811 00:13:14.977183 19902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:14.977522 19902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:14.977730 19902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:14.985018 19902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:14.985128 19902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:14.985348 19902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:13:14.995241 19902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:39205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.3.keytab
--principal=oryx/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:34285
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=46361
--tserver_master_addrs=127.7.94.62:36587,127.7.94.61:37589,127.7.94.60:39421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:14.996793 19902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:14.998222 19902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:15.011683 19914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:15.011734 19915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:15.011637 19918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:15.012835 19902 server_base.cc:1034] running on GCE node
I20260811 00:13:15.013509 19902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:15.014528 19902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:15.015725 19902 hybrid_clock.cc:648] HybridClock initialized: now 1786407195015689 us; error 35 us; skew 500 ppm
I20260811 00:13:15.016045 19902 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:15 dist-test-slave-8hhm krb5kdc[17825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407195, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:15.025630 19902 init.cc:377] Logged in from keytab as oryx/127.7.94.3@KRBTEST.COM (short username oryx)
I20260811 00:13:15.027292 19902 webserver.cc:460] Webserver started at http://127.7.94.3:46361/ using document root <none> and password file <none>
I20260811 00:13:15.028008 19902 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:15.028172 19902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:15.033723 19902 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:13:15.036506 19925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:15.037842 19902 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:13:15.038019 19902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "707ae6eda37543b087110d357cceadb8"
format_stamp: "Formatted at 2026-08-11 00:13:11 on dist-test-slave-8hhm"
server_key: "7eef5243ed906329bf38f5c885385988"
server_key_iv: "2576e09d1dd4d7a7e6a4fddd2957696a"
server_key_version: "encryptionkey@0"
I20260811 00:13:15.038998 19902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:15.059492 19902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:15.063486 19902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:15.063774 19902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:15.064746 19902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:15.066174 19902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:15.066306 19902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:15.066443 19902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:15.066532 19902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:15.101450 19902 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:34285
I20260811 00:13:15.101677 20131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:34285 every 8 connection(s)
I20260811 00:13:15.103004 19902 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:13:15.112358 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 19902
Aug 11 00:13:15 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407195, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.61@KRBTEST.COM
Aug 11 00:13:15 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407195, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.60@KRBTEST.COM
Aug 11 00:13:15 dist-test-slave-8hhm krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407195, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM
I20260811 00:13:15.166365 20157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.124521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60758 (local address 127.7.94.62:36587)
0811 00:13:15.125069 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:15.125078 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:15.142741 (+ 17663us) server_negotiation.cc:407] Connection header received
0811 00:13:15.142931 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:15.142954 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:15.143048 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:15.143157 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:15.147903 (+ 4746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.149087 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.151338 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.152044 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.152143 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.157670 (+ 5527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:15.157678 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:15.157683 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:15.157742 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:15.162801 (+ 5059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:15.163737 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:15.163742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:15.163745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:15.163897 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:15.164362 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:15.164367 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:15.164371 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:15.164780 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:15.165047 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:15.165699 (+ 652us) server_negotiation.cc:299] Negotiation successful
0811 00:13:15.165947 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":256,"threads_started":1}
I20260811 00:13:15.168123 20140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.106141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:41407 (local address 127.7.94.61:37589)
0811 00:13:15.121489 (+ 15348us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:15.121500 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:15.130012 (+ 8512us) server_negotiation.cc:407] Connection header received
0811 00:13:15.140335 (+ 10323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:15.140344 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:15.140467 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:15.140568 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:15.143188 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.144126 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.149669 (+ 5543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.150341 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.150447 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.159508 (+ 9061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:15.159518 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:15.159528 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:15.159574 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:15.164005 (+ 4431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:15.165033 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:15.165039 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:15.165045 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:15.165207 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:15.165812 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:15.165817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:15.165820 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:15.166161 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:15.166286 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:15.167334 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0811 00:13:15.167588 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15158,"thread_start_us":15361,"threads_started":1}
I20260811 00:13:15.169924 20158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.124606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52078 (local address 127.7.94.60:39421)
0811 00:13:15.128595 (+ 3989us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:15.128599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:15.135827 (+ 7228us) server_negotiation.cc:407] Connection header received
0811 00:13:15.135888 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:15.135894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:15.135982 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:15.136090 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:15.147865 (+ 11775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.150402 (+ 2537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.152407 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.153304 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.153400 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.160598 (+ 7198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:15.160610 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:15.160616 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:15.160666 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:15.166078 (+ 5412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:15.166859 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:15.166865 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:15.166868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:15.167061 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:15.167549 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:15.167555 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:15.167557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:15.167935 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:15.168224 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:15.169367 (+ 1143us) server_negotiation.cc:299] Negotiation successful
0811 00:13:15.169594 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3895,"thread_start_us":3778,"threads_started":1}
I20260811 00:13:15.170642 20145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.107372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:37589 (local address 127.7.94.3:41407)
0811 00:13:15.129886 (+ 22514us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:15.129910 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:15.133323 (+ 3413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:15.140884 (+ 7561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:15.140891 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:15.142682 (+ 1791us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:15.143008 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:15.143021 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.144292 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.144296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:15.149530 (+ 5234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:15.149541 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.150797 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.150803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:15.151133 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.152222 (+ 1089us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:15.152260 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:15.159211 (+ 6951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:15.164171 (+ 4960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:15.164182 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:15.164203 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:15.164887 (+ 684us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:15.165388 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:15.165394 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:15.165399 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:15.165677 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:15.166408 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:15.166417 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:15.166894 (+ 477us) client_negotiation.cc:769] Sending connection context
0811 00:13:15.168410 (+ 1516us) client_negotiation.cc:240] Negotiation successful
0811 00:13:15.169035 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22344,"mutex_wait_us":3,"thread_start_us":24006,"threads_started":1}
I20260811 00:13:15.172459 20138 heartbeater.cc:347] Connected to a master server at 127.7.94.61:37589
I20260811 00:13:15.172724 20167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.132038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:49393 (local address 127.7.94.62:36587)
0811 00:13:15.135641 (+ 3603us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:15.135649 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:15.139229 (+ 3580us) server_negotiation.cc:407] Connection header received
0811 00:13:15.140240 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:15.140248 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:15.140376 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:15.140488 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:15.145737 (+ 5249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.147118 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.149718 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.150585 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.150681 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.158532 (+ 7851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:15.158542 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:15.158549 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:15.158589 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:15.167027 (+ 8438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:15.170110 (+ 3083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:15.170116 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:15.170119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:15.170290 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:15.170765 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:15.170769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:15.170772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:15.171106 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:15.171243 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:15.171806 (+ 563us) server_negotiation.cc:299] Negotiation successful
0811 00:13:15.172198 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3438,"thread_start_us":3278,"threads_started":1}
I20260811 00:13:15.172816 20138 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:15.173796 20166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.131994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.3:49393)
0811 00:13:15.139101 (+ 7107us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:15.139126 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:15.140036 (+ 910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:15.140794 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:15.140802 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:15.141660 (+ 858us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:15.142051 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:15.142068 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.147302 (+ 5234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.147310 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:15.148945 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:15.148960 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.151113 (+ 2153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.151121 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:15.151427 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.154332 (+ 2905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:15.154359 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:15.158264 (+ 3905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:15.167689 (+ 9425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:15.167693 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:15.167696 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:15.168226 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:15.170438 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:15.170442 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:15.170445 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:15.170633 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:15.171327 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:15.171333 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:15.171479 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 00:13:15.172986 (+ 1507us) client_negotiation.cc:240] Negotiation successful
0811 00:13:15.173358 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6950,"mutex_wait_us":1,"thread_start_us":7252,"threads_started":1}
I20260811 00:13:15.173920 20138 heartbeater.cc:511] Master 127.7.94.61:37589 requested a full tablet report, sending...
I20260811 00:13:15.174717 20143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.106169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:47711 (local address 127.7.94.60:39421)
0811 00:13:15.122241 (+ 16072us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:15.122247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:15.129932 (+ 7685us) server_negotiation.cc:407] Connection header received
0811 00:13:15.134686 (+ 4754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:15.134695 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:15.134818 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:15.134916 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:15.143011 (+ 8095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.144590 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.149481 (+ 4891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.150115 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.150240 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.158312 (+ 8072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:15.158320 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:15.158326 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:15.158372 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:15.167539 (+ 9167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:15.170361 (+ 2822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:15.170366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:15.170369 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:15.170523 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:15.170969 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:15.170987 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:15.170992 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:15.171359 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:15.171484 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:15.174049 (+ 2565us) server_negotiation.cc:299] Negotiation successful
0811 00:13:15.174321 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15886,"thread_start_us":15461,"threads_started":1}
I20260811 00:13:15.175729 20135 heartbeater.cc:347] Connected to a master server at 127.7.94.62:36587
I20260811 00:13:15.175885 20135 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:15.176050 18969 ts_manager.cc:194] Registered new tserver with Master: 707ae6eda37543b087110d357cceadb8 (127.7.94.3:34285)
I20260811 00:13:15.176518 20135 heartbeater.cc:511] Master 127.7.94.62:36587 requested a full tablet report, sending...
I20260811 00:13:15.175734 20146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.106663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:39421 (local address 127.7.94.3:47711)
0811 00:13:15.129671 (+ 23008us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:15.129749 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:15.133662 (+ 3913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:15.135556 (+ 1894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:15.135568 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:15.136760 (+ 1192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:15.141741 (+ 4981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:15.141779 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.144884 (+ 3105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.144892 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:15.147261 (+ 2369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:15.147282 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.151628 (+ 4346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.151634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:15.151813 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.154216 (+ 2403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:15.154258 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:15.157957 (+ 3699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:15.168342 (+ 10385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:15.168345 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:15.168349 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:15.168884 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:15.170669 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:15.170674 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:15.170678 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:15.170816 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:15.171574 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:15.171579 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:15.171756 (+ 177us) client_negotiation.cc:769] Sending connection context
0811 00:13:15.174677 (+ 2921us) client_negotiation.cc:240] Negotiation successful
0811 00:13:15.175143 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22456,"thread_start_us":23154,"threads_started":1}
I20260811 00:13:15.177928 17913 ts_manager.cc:194] Re-registered known tserver with Master: 707ae6eda37543b087110d357cceadb8 (127.7.94.3:34285)
I20260811 00:13:15.177989 20132 heartbeater.cc:347] Connected to a master server at 127.7.94.60:39421
I20260811 00:13:15.178112 20132 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:15.178534 20132 heartbeater.cc:511] Master 127.7.94.60:39421 requested a full tablet report, sending...
I20260811 00:13:15.179371 17913 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:49393
I20260811 00:13:15.179658 19317 ts_manager.cc:194] Registered new tserver with Master: 707ae6eda37543b087110d357cceadb8 (127.7.94.3:34285)
I20260811 00:13:15.183670 20168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.135960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59074 (local address 127.7.94.61:37589)
0811 00:13:15.139904 (+ 3944us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:15.139913 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:15.149618 (+ 9705us) server_negotiation.cc:407] Connection header received
0811 00:13:15.152349 (+ 2731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:15.152354 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:15.152447 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:15.152544 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:15.155876 (+ 3332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.157144 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.161663 (+ 4519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.162483 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.162581 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.176025 (+ 13444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:15.176036 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:15.176040 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:15.176077 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:15.180687 (+ 4610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:15.181514 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:15.181519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:15.181522 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:15.181766 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:15.182183 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:15.182186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:15.182188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:15.182497 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:15.182693 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:15.183179 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 00:13:15.183337 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3774,"thread_start_us":3632,"threads_started":1}
I20260811 00:13:15.174247 17914 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60758:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:13:15.185644 17914 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:13:15.235642 20190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.205023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:36059 (local address 127.0.0.1:55932)
0811 00:13:15.221631 (+ 16608us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:15.221651 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:15.221782 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:15.226595 (+ 4813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:15.226603 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:15.226637 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:15.227351 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:15.227364 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.230017 (+ 2653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.230025 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:15.232737 (+ 2712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:15.232766 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.234845 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.234851 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:15.235041 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.235082 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 00:13:15.235162 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 00:13:15.235278 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16431,"thread_start_us":15964,"threads_started":1}
I20260811 00:13:15.236270 20191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.205665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55932 (local address 127.7.94.1:36059)
0811 00:13:15.225720 (+ 20055us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:15.225733 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:15.225774 (+ 41us) server_negotiation.cc:407] Connection header received
0811 00:13:15.225908 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:15.225920 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:15.226279 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:15.227626 (+ 1347us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:15.227805 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.229684 (+ 1879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.233036 (+ 3352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.234660 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.235329 (+ 669us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.235560 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:15.235644 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 00:13:15.235932 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19861,"thread_start_us":19729,"threads_started":1}
I20260811 00:13:15.237267 20188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.204908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43092 (local address 127.7.94.3:34285)
0811 00:13:15.227251 (+ 22343us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:15.227262 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:15.227302 (+ 40us) server_negotiation.cc:407] Connection header received
0811 00:13:15.227427 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:15.227439 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:15.227760 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:15.227994 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:15.229007 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.231157 (+ 2150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.233950 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.235426 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.235565 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.235759 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:15.236733 (+ 974us) server_negotiation.cc:299] Negotiation successful
0811 00:13:15.236898 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22119,"thread_start_us":21884,"threads_started":1}
I20260811 00:13:15.237306 20189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.204485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:34285 (local address 127.0.0.1:43092)
0811 00:13:15.220514 (+ 16029us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:15.220542 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:15.220708 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:15.228006 (+ 7298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:15.228011 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:15.228037 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:15.228767 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:15.228785 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.231386 (+ 2601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.231391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:15.233733 (+ 2342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:15.233765 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.236323 (+ 2558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.236328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:15.236526 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.236564 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 00:13:15.236686 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 00:13:15.236809 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15805,"thread_start_us":26286,"threads_started":1}
I20260811 00:13:15.240207 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.205858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52618 (local address 127.7.94.2:35337)
0811 00:13:15.228697 (+ 22839us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:15.228709 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:15.231261 (+ 2552us) server_negotiation.cc:407] Connection header received
0811 00:13:15.231432 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:15.231443 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:15.231795 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:15.231995 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:15.233460 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.234986 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.238432 (+ 3446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.239290 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.239409 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.239537 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:15.239897 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 00:13:15.240020 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22638,"thread_start_us":22512,"threads_started":1}
I20260811 00:13:15.240636 20186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.205687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:35337 (local address 127.0.0.1:52618)
0811 00:13:15.231086 (+ 25399us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:15.231123 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:15.231259 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:15.232039 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:15.232044 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:15.232069 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:15.233159 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:15.233183 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.235836 (+ 2653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.235841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:15.238241 (+ 2400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:15.238260 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.239526 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.239533 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:15.239712 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.239746 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 00:13:15.239836 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 00:13:15.239957 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25269,"mutex_wait_us":1,"thread_start_us":15616,"threads_started":1}
I20260811 00:13:15.240559 19522 tablet_service.cc:1467] Processing CreateTablet for tablet 040b8d5b6b764964b0255112e2754a15 (DEFAULT_TABLE table=test-table [id=03dbb9ef219a469680cb40d2168f547e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:15.242784 19522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 040b8d5b6b764964b0255112e2754a15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:15.244050 20017 tablet_service.cc:1467] Processing CreateTablet for tablet 040b8d5b6b764964b0255112e2754a15 (DEFAULT_TABLE table=test-table [id=03dbb9ef219a469680cb40d2168f547e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:15.244355 19766 tablet_service.cc:1467] Processing CreateTablet for tablet 040b8d5b6b764964b0255112e2754a15 (DEFAULT_TABLE table=test-table [id=03dbb9ef219a469680cb40d2168f547e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:15.245247 20017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 040b8d5b6b764964b0255112e2754a15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:15.246104 19766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 040b8d5b6b764964b0255112e2754a15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:15.258569 20196 tablet_bootstrap.cc:492] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8: Bootstrap starting.
I20260811 00:13:15.261471 20196 tablet_bootstrap.cc:654] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:15.262063 20198 tablet_bootstrap.cc:492] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935: Bootstrap starting.
I20260811 00:13:15.263193 20196 log.cc:824] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8: Log is configured to *not* fsync() on all Append() calls
I20260811 00:13:15.263444 20201 tablet_bootstrap.cc:492] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c: Bootstrap starting.
I20260811 00:13:15.266422 20198 tablet_bootstrap.cc:654] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:15.266741 20201 tablet_bootstrap.cc:654] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:15.267450 20196 tablet_bootstrap.cc:492] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8: No bootstrap required, opened a new log
I20260811 00:13:15.267735 20196 ts_tablet_manager.cc:1397] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.006s
I20260811 00:13:15.267774 20201 log.cc:824] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c: Log is configured to *not* fsync() on all Append() calls
I20260811 00:13:15.267887 20198 log.cc:824] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935: Log is configured to *not* fsync() on all Append() calls
I20260811 00:13:15.276294 20201 tablet_bootstrap.cc:492] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c: No bootstrap required, opened a new log
I20260811 00:13:15.276625 20201 ts_tablet_manager.cc:1397] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260811 00:13:15.277396 20198 tablet_bootstrap.cc:492] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935: No bootstrap required, opened a new log
I20260811 00:13:15.277666 20198 ts_tablet_manager.cc:1397] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.004s
I20260811 00:13:15.280385 20196 raft_consensus.cc:348] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 } } peers { permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 } } peers { permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 } }
I20260811 00:13:15.280813 20196 raft_consensus.cc:374] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:15.280917 20196 raft_consensus.cc:729] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 707ae6eda37543b087110d357cceadb8, State: Initialized, Role: FOLLOWER
I20260811 00:13:15.281713 20196 consensus_queue.cc:260] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 } } peers { permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 } } peers { permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 } }
I20260811 00:13:15.285034 20201 raft_consensus.cc:348] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 } } peers { permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 } } peers { permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 } }
I20260811 00:13:15.285465 20201 raft_consensus.cc:374] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:15.285604 20201 raft_consensus.cc:729] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6905d72f434b7f86d5d92f6a46184c, State: Initialized, Role: FOLLOWER
I20260811 00:13:15.286405 20201 consensus_queue.cc:260] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 } } peers { permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 } } peers { permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 } }
I20260811 00:13:15.287061 20196 ts_tablet_manager.cc:1428] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8: Time spent starting tablet: real 0.019s user 0.017s sys 0.003s
I20260811 00:13:15.287330 20135 heartbeater.cc:503] Master 127.7.94.62:36587 was elected leader, sending a full tablet report...
I20260811 00:13:15.289175 19879 heartbeater.cc:503] Master 127.7.94.62:36587 was elected leader, sending a full tablet report...
I20260811 00:13:15.289281 20198 raft_consensus.cc:348] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 } } peers { permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 } } peers { permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 } }
I20260811 00:13:15.289682 20198 raft_consensus.cc:374] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:15.289786 20198 raft_consensus.cc:729] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c488dd19f66d45419d4054c024614935, State: Initialized, Role: FOLLOWER
I20260811 00:13:15.290728 20198 consensus_queue.cc:260] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 } } peers { permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 } } peers { permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 } }
I20260811 00:13:15.291849 20201 ts_tablet_manager.cc:1428] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c: Time spent starting tablet: real 0.015s user 0.009s sys 0.005s
I20260811 00:13:15.298069 20211 raft_consensus.cc:482] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:13:15.298411 20211 raft_consensus.cc:504] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 } } peers { permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 } } peers { permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 } }
I20260811 00:13:15.301198 20211 leader_election.cc:288] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 707ae6eda37543b087110d357cceadb8 (127.7.94.3:34285), 6c6905d72f434b7f86d5d92f6a46184c (127.7.94.2:35337)
I20260811 00:13:15.302284 20198 ts_tablet_manager.cc:1428] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935: Time spent starting tablet: real 0.024s user 0.014s sys 0.005s
W20260811 00:13:15.306617 19637 tablet.cc:2369] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:13:15.308511 20188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.301068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:36355 (local address 127.7.94.3:34285)
0811 00:13:15.301317 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:15.301326 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:15.302115 (+ 789us) server_negotiation.cc:407] Connection header received
0811 00:13:15.302294 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:15.302302 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:15.302398 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:15.302493 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:15.303220 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.304060 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.306385 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.307541 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.307715 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.307841 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:15.308139 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 00:13:15.308265 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 00:13:15.308671 20214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.300992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:34285 (local address 127.7.94.1:36355)
0811 00:13:15.301987 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:15.302009 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:15.302147 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:15.302527 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:15.302531 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:15.302551 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:15.302961 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:15.302976 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.304250 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.304255 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:15.306188 (+ 1933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:15.306204 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.307812 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.307825 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:15.307976 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.308003 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 00:13:15.308114 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 00:13:15.308257 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":3,"thread_start_us":507,"threads_started":1}
I20260811 00:13:15.309573 20045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "040b8d5b6b764964b0255112e2754a15" candidate_uuid: "c488dd19f66d45419d4054c024614935" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "707ae6eda37543b087110d357cceadb8" is_pre_election: true
I20260811 00:13:15.310052 20045 raft_consensus.cc:2454] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c488dd19f66d45419d4054c024614935 in term 0.
I20260811 00:13:15.310772 19446 leader_election.cc:302] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 707ae6eda37543b087110d357cceadb8, c488dd19f66d45419d4054c024614935; no voters:
I20260811 00:13:15.311172 20211 raft_consensus.cc:2781] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:13:15.311321 20211 raft_consensus.cc:482] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:13:15.311457 20211 raft_consensus.cc:3037] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:15.311662 20195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.301896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:41761 (local address 127.7.94.2:35337)
0811 00:13:15.302066 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:15.302072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:15.305614 (+ 3542us) server_negotiation.cc:407] Connection header received
0811 00:13:15.305764 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:15.305769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:15.305839 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:15.305926 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:15.306502 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.307326 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.309420 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.310661 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.310818 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.310932 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:15.311265 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 00:13:15.311393 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 00:13:15.311816 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.301640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:35337 (local address 127.7.94.1:41761)
0811 00:13:15.305479 (+ 3839us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:15.305494 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:15.305633 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:15.305955 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:15.305958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:15.305976 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:15.306327 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:15.306334 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.307469 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.307474 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:15.309265 (+ 1791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:15.309282 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.310937 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.310946 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:15.311103 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.311126 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 00:13:15.311238 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 00:13:15.311363 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3721,"mutex_wait_us":4,"thread_start_us":778,"threads_started":1}
I20260811 00:13:15.312813 19805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "040b8d5b6b764964b0255112e2754a15" candidate_uuid: "c488dd19f66d45419d4054c024614935" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6905d72f434b7f86d5d92f6a46184c" is_pre_election: true
I20260811 00:13:15.313386 19805 raft_consensus.cc:2454] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c488dd19f66d45419d4054c024614935 in term 0.
I20260811 00:13:15.314425 20211 raft_consensus.cc:504] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 } } peers { permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 } } peers { permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 } }
I20260811 00:13:15.315030 20211 leader_election.cc:288] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [CANDIDATE]: Term 1 election: Requested vote from peers 707ae6eda37543b087110d357cceadb8 (127.7.94.3:34285), 6c6905d72f434b7f86d5d92f6a46184c (127.7.94.2:35337)
I20260811 00:13:15.315472 20045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "040b8d5b6b764964b0255112e2754a15" candidate_uuid: "c488dd19f66d45419d4054c024614935" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "707ae6eda37543b087110d357cceadb8"
I20260811 00:13:15.315546 19805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "040b8d5b6b764964b0255112e2754a15" candidate_uuid: "c488dd19f66d45419d4054c024614935" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6905d72f434b7f86d5d92f6a46184c"
I20260811 00:13:15.315663 20045 raft_consensus.cc:3037] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:15.315745 19805 raft_consensus.cc:3037] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:15.318611 20045 raft_consensus.cc:2454] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c488dd19f66d45419d4054c024614935 in term 1.
I20260811 00:13:15.318600 19805 raft_consensus.cc:2454] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c488dd19f66d45419d4054c024614935 in term 1.
I20260811 00:13:15.319080 19450 leader_election.cc:302] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c6905d72f434b7f86d5d92f6a46184c, c488dd19f66d45419d4054c024614935; no voters:
I20260811 00:13:15.319458 20211 raft_consensus.cc:2781] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:13:15.319628 20211 raft_consensus.cc:686] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [term 1 LEADER]: Becoming Leader. State: Replica: c488dd19f66d45419d4054c024614935, State: Running, Role: LEADER
I20260811 00:13:15.320165 20211 consensus_queue.cc:237] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 } } peers { permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 } } peers { permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 } }
I20260811 00:13:15.325670 17914 catalog_manager.cc:5495] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 reported cstate change: term changed from 0 to 1, leader changed from <none> to c488dd19f66d45419d4054c024614935 (127.7.94.1). New cstate: current_term: 1 leader_uuid: "c488dd19f66d45419d4054c024614935" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 } health_report { overall_health: UNKNOWN } } }
I20260811 00:13:15.354315 20191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.349061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55936 (local address 127.7.94.1:36059)
0811 00:13:15.349310 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:15.349316 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:15.349506 (+ 190us) server_negotiation.cc:407] Connection header received
0811 00:13:15.349668 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:15.349671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:15.349747 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:15.349867 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:13:15.350198 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.350921 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.352296 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.352697 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.352787 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.353207 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:13:15.353709 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:13:15.353813 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:15.353921 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 00:13:15.354001 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 00:13:15.361557 20139 tablet.cc:2369] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:13:15.362319 19805 raft_consensus.cc:1261] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c [term 1 FOLLOWER]: Refusing update from remote peer c488dd19f66d45419d4054c024614935: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:13:15.362444 20045 raft_consensus.cc:1261] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8 [term 1 FOLLOWER]: Refusing update from remote peer c488dd19f66d45419d4054c024614935: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:13:15.363399 20212 consensus_queue.cc:1035] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [LEADER]: Connected to new peer: Peer: permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:13:15.363780 20211 consensus_queue.cc:1035] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:13:15.380539 20221 mvcc.cc:205] Tried to move back new op lower bound from 7317123872194162688 to 7317123872035364864. Current Snapshot: MvccSnapshot[applied={T|T < 7317123872194162688}]
I20260811 00:13:15.383497 20225 mvcc.cc:205] Tried to move back new op lower bound from 7317123872194162688 to 7317123872035364864. Current Snapshot: MvccSnapshot[applied={T|T < 7317123872194162688}]
I20260811 00:13:15.401623 17914 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:13:15.401952 17914 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:13:15.408231 17914 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aad6ba1a2f694580b8fa3dc892b1d83b: Sending DeleteTablet for 3 replicas of tablet 040b8d5b6b764964b0255112e2754a15
I20260811 00:13:15.409495 20017 tablet_service.cc:1514] Processing DeleteTablet for tablet 040b8d5b6b764964b0255112e2754a15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:15 UTC) from {username='oryx', principal='oryx/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:43092
I20260811 00:13:15.409750 19766 tablet_service.cc:1514] Processing DeleteTablet for tablet 040b8d5b6b764964b0255112e2754a15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:15 UTC) from {username='oryx', principal='oryx/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:52618
I20260811 00:13:15.411584 19522 tablet_service.cc:1514] Processing DeleteTablet for tablet 040b8d5b6b764964b0255112e2754a15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:15 UTC) from {username='oryx', principal='oryx/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:55932
I20260811 00:13:15.413875 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 17852
I20260811 00:13:15.414964 17968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:15.435576 20242 tablet_replica.cc:330] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935: stopping tablet replica
I20260811 00:13:15.436290 20239 tablet_replica.cc:330] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8: stopping tablet replica
I20260811 00:13:15.437178 20242 raft_consensus.cc:2229] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:13:15.437975 20239 raft_consensus.cc:2229] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:13:15.438553 20242 raft_consensus.cc:2258] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:15.439025 20168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.411502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59076 (local address 127.7.94.61:37589)
0811 00:13:15.411685 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:15.411691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:15.413011 (+ 1320us) server_negotiation.cc:407] Connection header received
0811 00:13:15.414541 (+ 1530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:15.414548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:15.414668 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:15.414767 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:15.418616 (+ 3849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.420115 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.422699 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.423611 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.423736 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.428610 (+ 4874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:15.428620 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:15.428627 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:15.428681 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:15.434029 (+ 5348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:15.435212 (+ 1183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:15.435221 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:15.435227 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:15.435451 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:15.436228 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:15.436236 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:15.436242 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:15.436820 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:15.437139 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:15.438006 (+ 867us) server_negotiation.cc:299] Negotiation successful
0811 00:13:15.438432 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 00:13:15.439435 20239 raft_consensus.cc:2258] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:15.441903 20242 ts_tablet_manager.cc:1905] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:15.445878 20240 tablet_replica.cc:330] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c: stopping tablet replica
I20260811 00:13:15.446358 20240 raft_consensus.cc:2229] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:13:15.447093 20240 raft_consensus.cc:2258] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:15.448892 20239 ts_tablet_manager.cc:1905] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:15.456312 20239 ts_tablet_manager.cc:1918] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:13:15.456499 20239 log.cc:1196] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/040b8d5b6b764964b0255112e2754a15
I20260811 00:13:15.457080 20239 ts_tablet_manager.cc:1939] T 040b8d5b6b764964b0255112e2754a15 P 707ae6eda37543b087110d357cceadb8: Deleting consensus metadata
I20260811 00:13:15.458405 20143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.412398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52094 (local address 127.7.94.60:39421)
0811 00:13:15.412577 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:15.412586 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:15.440435 (+ 27849us) server_negotiation.cc:407] Connection header received
0811 00:13:15.440706 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:15.440713 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:15.440834 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:15.440911 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:15.443505 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.445051 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.446741 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.447245 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.447384 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.450643 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:15.450650 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:15.450655 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:15.450704 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:15.454609 (+ 3905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:15.455531 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:15.455537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:15.455540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:15.455700 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:15.456278 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:15.456283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:15.456287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:15.456762 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:15.457028 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:15.457739 (+ 711us) server_negotiation.cc:299] Negotiation successful
0811 00:13:15.458009 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 00:13:15.458964 20240 ts_tablet_manager.cc:1905] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:15.464918 20242 ts_tablet_manager.cc:1918] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:13:15.465116 20242 log.cc:1196] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/040b8d5b6b764964b0255112e2754a15
I20260811 00:13:15.465783 20242 ts_tablet_manager.cc:1939] T 040b8d5b6b764964b0255112e2754a15 P c488dd19f66d45419d4054c024614935: Deleting consensus metadata
I20260811 00:13:15.468966 20240 ts_tablet_manager.cc:1918] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:13:15.469137 20240 log.cc:1196] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/040b8d5b6b764964b0255112e2754a15
I20260811 00:13:15.469802 20240 ts_tablet_manager.cc:1939] T 040b8d5b6b764964b0255112e2754a15 P 6c6905d72f434b7f86d5d92f6a46184c: Deleting consensus metadata
I20260811 00:13:15.584945 17884 catalog_manager.cc:4841] TS 6c6905d72f434b7f86d5d92f6a46184c (127.7.94.2:35337): tablet 040b8d5b6b764964b0255112e2754a15 (table test-table [id=03dbb9ef219a469680cb40d2168f547e]) successfully deleted
I20260811 00:13:15.586246 17881 catalog_manager.cc:4841] TS 707ae6eda37543b087110d357cceadb8 (127.7.94.3:34285): tablet 040b8d5b6b764964b0255112e2754a15 (table test-table [id=03dbb9ef219a469680cb40d2168f547e]) successfully deleted
I20260811 00:13:15.586273 17882 catalog_manager.cc:4841] TS c488dd19f66d45419d4054c024614935 (127.7.94.1:36059): tablet 040b8d5b6b764964b0255112e2754a15 (table test-table [id=03dbb9ef219a469680cb40d2168f547e]) successfully deleted
I20260811 00:13:15.587023 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 17852
I20260811 00:13:15.647264 20143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.629638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52108 (local address 127.7.94.60:39421)
0811 00:13:15.629831 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:15.629839 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:15.630829 (+ 990us) server_negotiation.cc:407] Connection header received
0811 00:13:15.631110 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:15.631117 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:15.631203 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:15.631279 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:15.633882 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.634747 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.636693 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.637364 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.637499 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.640472 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:15.640478 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:15.640483 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:15.640525 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:15.643875 (+ 3350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:15.644713 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:15.644719 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:15.644723 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:15.644930 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:15.645353 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:15.645358 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:15.645362 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:15.645770 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:15.646045 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:15.646647 (+ 602us) server_negotiation.cc:299] Negotiation successful
0811 00:13:15.646874 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:13:15.647553 20168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:15.630661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59088 (local address 127.7.94.61:37589)
0811 00:13:15.630906 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:15.630916 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:15.631978 (+ 1062us) server_negotiation.cc:407] Connection header received
0811 00:13:15.632176 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:15.632182 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:15.632255 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:15.632376 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:15.634210 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.635360 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.637026 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:15.637596 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:15.637738 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:15.641243 (+ 3505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:15.641252 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:15.641260 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:15.641305 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:15.644772 (+ 3467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:15.645552 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:15.645557 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:15.645560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:15.645703 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:15.646090 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:15.646095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:15.646098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:15.646372 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:15.646627 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:15.646940 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 00:13:15.647147 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 00:13:16.391554 20255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:16.390461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.2:49435)
0811 00:13:16.391191 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:16.391338 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":505,"threads_started":1}
W20260811 00:13:16.392210 19879 heartbeater.cc:644] Failed to heartbeat to 127.7.94.62:36587 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
I20260811 00:13:16.397737 20256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:16.396729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.3:60953)
0811 00:13:16.397321 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:16.397469 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":285,"threads_started":1}
W20260811 00:13:16.398265 20135 heartbeater.cc:644] Failed to heartbeat to 127.7.94.62:36587 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
I20260811 00:13:16.398910 20257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:16.398088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.1:42967)
0811 00:13:16.398640 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:16.398763 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":309,"threads_started":1}
W20260811 00:13:16.399274 19634 heartbeater.cc:644] Failed to heartbeat to 127.7.94.62:36587 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
I20260811 00:13:16.909195 20258 raft_consensus.cc:482] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aad6ba1a2f694580b8fa3dc892b1d83b)
I20260811 00:13:16.909554 20258 raft_consensus.cc:504] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false } }
I20260811 00:13:16.910457 20264 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aad6ba1a2f694580b8fa3dc892b1d83b)
I20260811 00:13:16.910701 20264 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false } }
I20260811 00:13:16.911164 20258 leader_election.cc:288] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aad6ba1a2f694580b8fa3dc892b1d83b (127.7.94.62:36587), f2fdaa9fb639489a89d8ff8c2bb0dfb4 (127.7.94.61:37589)
I20260811 00:13:16.912300 20264 leader_election.cc:288] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aad6ba1a2f694580b8fa3dc892b1d83b (127.7.94.62:36587), a124e8d010f24686a8793e2134a7febc (127.7.94.60:39421)
I20260811 00:13:16.912225 20265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:16.911339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:32986)
0811 00:13:16.911804 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:16.911984 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":235,"threads_started":1}
W20260811 00:13:16.913830 19298 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
I20260811 00:13:16.915505 20273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:16.912323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:33002)
0811 00:13:16.914224 (+ 1901us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:16.914994 (+ 770us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1813,"thread_start_us":1538,"threads_started":1}
W20260811 00:13:16.915997 18950 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
I20260811 00:13:16.916184 20265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:16.915756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:33010)
0811 00:13:16.915935 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:16.916019 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 00:13:16.916491 19298 leader_election.cc:334] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aad6ba1a2f694580b8fa3dc892b1d83b (127.7.94.62:36587): Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
I20260811 00:13:16.918841 20277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:16.918259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:33016)
0811 00:13:16.918460 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:16.918602 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 00:13:16.919052 18950 leader_election.cc:334] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aad6ba1a2f694580b8fa3dc892b1d83b (127.7.94.62:36587): Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
I20260811 00:13:16.923017 20273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:16.913348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:39421 (local address 127.0.0.1:52116)
0811 00:13:16.915856 (+ 2508us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:16.915874 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:16.915992 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:16.916579 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:16.916583 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:16.916611 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:16.917528 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:16.917546 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:16.918818 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:16.918824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:16.921002 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:16.921015 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:16.922249 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:16.922255 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:16.922435 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:16.922485 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 00:13:16.922576 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 00:13:16.922710 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2403,"thread_start_us":3860,"threads_started":1}
I20260811 00:13:16.923130 20276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:16.913009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52116 (local address 127.7.94.60:39421)
0811 00:13:16.913541 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:16.913546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:16.916099 (+ 2553us) server_negotiation.cc:407] Connection header received
0811 00:13:16.916174 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:16.916180 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:16.916253 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:16.916349 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:16.917775 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:16.918680 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:16.921336 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:16.922078 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:16.922225 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:16.922313 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:16.922620 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 00:13:16.922747 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":554,"threads_started":1}
I20260811 00:13:16.923243 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:16.911994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:37589 (local address 127.0.0.1:59096)
0811 00:13:16.912911 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:16.912935 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:16.913094 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:16.914411 (+ 1317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:16.914416 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:16.914442 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:16.914709 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:16.914717 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:16.918921 (+ 4204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:16.918928 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:16.920996 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:16.921012 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:16.922395 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:16.922401 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:16.922589 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:16.922612 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 00:13:16.922723 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 00:13:16.922853 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"thread_start_us":616,"threads_started":1}
I20260811 00:13:16.923684 20272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:16.912099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59096 (local address 127.7.94.61:37589)
0811 00:13:16.914011 (+ 1912us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:16.914017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:16.914038 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:13:16.914112 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:16.914117 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:16.914285 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:16.914387 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:16.916862 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:16.918697 (+ 1835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:16.921172 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:16.922203 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:16.922354 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:16.922464 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:16.922747 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 00:13:16.922876 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1797,"thread_start_us":1430,"threads_started":1}
I20260811 00:13:16.923938 19334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a124e8d010f24686a8793e2134a7febc" is_pre_election: true
I20260811 00:13:16.924273 19334 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2fdaa9fb639489a89d8ff8c2bb0dfb4 in term 1.
I20260811 00:13:16.924242 18986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a124e8d010f24686a8793e2134a7febc" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" is_pre_election: true
I20260811 00:13:16.924527 18986 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a124e8d010f24686a8793e2134a7febc in term 1.
I20260811 00:13:16.924785 18949 leader_election.cc:302] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a124e8d010f24686a8793e2134a7febc, f2fdaa9fb639489a89d8ff8c2bb0dfb4; no voters: aad6ba1a2f694580b8fa3dc892b1d83b
I20260811 00:13:16.924990 19298 leader_election.cc:302] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a124e8d010f24686a8793e2134a7febc, f2fdaa9fb639489a89d8ff8c2bb0dfb4; no voters: aad6ba1a2f694580b8fa3dc892b1d83b
I20260811 00:13:16.925227 20264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 00:13:16.925359 20264 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader aad6ba1a2f694580b8fa3dc892b1d83b)
I20260811 00:13:16.925462 20264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 1 FOLLOWER]: Advancing to term 2
I20260811 00:13:16.925412 20258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 00:13:16.925578 20258 raft_consensus.cc:482] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 1 FOLLOWER]: Starting leader election (detected failure of leader aad6ba1a2f694580b8fa3dc892b1d83b)
I20260811 00:13:16.925678 20258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 1 FOLLOWER]: Advancing to term 2
I20260811 00:13:16.927788 20258 raft_consensus.cc:504] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false } }
I20260811 00:13:16.927798 20264 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false } }
I20260811 00:13:16.928428 20258 leader_election.cc:288] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [CANDIDATE]: Term 2 election: Requested vote from peers aad6ba1a2f694580b8fa3dc892b1d83b (127.7.94.62:36587), f2fdaa9fb639489a89d8ff8c2bb0dfb4 (127.7.94.61:37589)
I20260811 00:13:16.928458 20264 leader_election.cc:288] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [CANDIDATE]: Term 2 election: Requested vote from peers aad6ba1a2f694580b8fa3dc892b1d83b (127.7.94.62:36587), a124e8d010f24686a8793e2134a7febc (127.7.94.60:39421)
I20260811 00:13:16.929281 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:16.928841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:33040)
0811 00:13:16.929020 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:16.929104 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 00:13:16.929411 18986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a124e8d010f24686a8793e2134a7febc" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4"
I20260811 00:13:16.929607 20273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:16.928526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:33022)
0811 00:13:16.929153 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:16.929253 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514}
I20260811 00:13:16.929733 18986 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a124e8d010f24686a8793e2134a7febc in current term 2: Already voted for candidate f2fdaa9fb639489a89d8ff8c2bb0dfb4 in this term.
I20260811 00:13:16.930430 19334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a124e8d010f24686a8793e2134a7febc"
I20260811 00:13:16.930744 19334 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f2fdaa9fb639489a89d8ff8c2bb0dfb4 in current term 2: Already voted for candidate a124e8d010f24686a8793e2134a7febc in this term.
I20260811 00:13:16.930939 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:16.929864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:33042)
0811 00:13:16.930606 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:16.930722 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":635}
W20260811 00:13:16.931113 19298 leader_election.cc:334] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer aad6ba1a2f694580b8fa3dc892b1d83b (127.7.94.62:36587): Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
I20260811 00:13:16.931254 20277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:16.930876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:33054)
0811 00:13:16.931016 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:16.931100 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 00:13:16.931247 19298 leader_election.cc:302] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a124e8d010f24686a8793e2134a7febc; no voters: aad6ba1a2f694580b8fa3dc892b1d83b, f2fdaa9fb639489a89d8ff8c2bb0dfb4
I20260811 00:13:16.931499 20258 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260811 00:13:16.931476 18950 leader_election.cc:334] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer aad6ba1a2f694580b8fa3dc892b1d83b (127.7.94.62:36587): Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
I20260811 00:13:16.931914 18949 leader_election.cc:302] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2fdaa9fb639489a89d8ff8c2bb0dfb4; no voters: a124e8d010f24686a8793e2134a7febc, aad6ba1a2f694580b8fa3dc892b1d83b
I20260811 00:13:16.932173 20264 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 00:13:17.393882 20283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:17.392894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.2:45899)
0811 00:13:17.393528 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:17.393636 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":394,"threads_started":1}
I20260811 00:13:17.399760 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:17.398915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.3:48835)
0811 00:13:17.399404 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:17.399508 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":267,"threads_started":1}
I20260811 00:13:17.400715 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:17.399949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.1:42151)
0811 00:13:17.400430 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:17.400539 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":291,"threads_started":1}
I20260811 00:13:18.395884 20289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.394905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.2:40003)
0811 00:13:18.395435 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:18.395605 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":289,"threads_started":1}
I20260811 00:13:18.402191 20293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.401414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.1:41919)
0811 00:13:18.401872 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:18.401984 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":274,"threads_started":1}
I20260811 00:13:18.405774 20290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.400631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.3:37549)
0811 00:13:18.405560 (+ 4929us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:18.405632 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4839,"thread_start_us":4730,"threads_started":1}
I20260811 00:13:18.853588 20294 raft_consensus.cc:482] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:13:18.853827 20294 raft_consensus.cc:504] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false } }
I20260811 00:13:18.854458 20294 leader_election.cc:288] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aad6ba1a2f694580b8fa3dc892b1d83b (127.7.94.62:36587), f2fdaa9fb639489a89d8ff8c2bb0dfb4 (127.7.94.61:37589)
I20260811 00:13:18.855481 20295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.854638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:33294)
0811 00:13:18.855207 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:18.855315 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":280,"threads_started":1}
I20260811 00:13:18.855666 18986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a124e8d010f24686a8793e2134a7febc" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" is_pre_election: true
I20260811 00:13:18.855934 18986 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a124e8d010f24686a8793e2134a7febc in term 2.
I20260811 00:13:18.856413 19298 leader_election.cc:302] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a124e8d010f24686a8793e2134a7febc, f2fdaa9fb639489a89d8ff8c2bb0dfb4; no voters:
I20260811 00:13:18.856691 20294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 00:13:18.856846 20294 raft_consensus.cc:482] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:13:18.856956 20294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 2 FOLLOWER]: Advancing to term 3
I20260811 00:13:18.857335 20295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.856805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:33296)
0811 00:13:18.856991 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:18.857130 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 00:13:18.857563 19298 leader_election.cc:334] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer aad6ba1a2f694580b8fa3dc892b1d83b (127.7.94.62:36587): Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
I20260811 00:13:18.859273 20294 raft_consensus.cc:504] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false } }
I20260811 00:13:18.859844 20294 leader_election.cc:288] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [CANDIDATE]: Term 3 election: Requested vote from peers aad6ba1a2f694580b8fa3dc892b1d83b (127.7.94.62:36587), f2fdaa9fb639489a89d8ff8c2bb0dfb4 (127.7.94.61:37589)
I20260811 00:13:18.860309 20295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.859988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:33304)
0811 00:13:18.860098 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:18.860193 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 00:13:18.860555 18986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a124e8d010f24686a8793e2134a7febc" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4"
I20260811 00:13:18.860754 18986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 2 FOLLOWER]: Advancing to term 3
I20260811 00:13:18.861294 20295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.860880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:33312)
0811 00:13:18.861050 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:18.861136 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 00:13:18.861452 19298 leader_election.cc:334] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer aad6ba1a2f694580b8fa3dc892b1d83b (127.7.94.62:36587): Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
I20260811 00:13:18.863036 18986 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a124e8d010f24686a8793e2134a7febc in term 3.
I20260811 00:13:18.863440 19298 leader_election.cc:302] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a124e8d010f24686a8793e2134a7febc, f2fdaa9fb639489a89d8ff8c2bb0dfb4; no voters: aad6ba1a2f694580b8fa3dc892b1d83b
I20260811 00:13:18.863818 20294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 3 FOLLOWER]: Leader election won for term 3
I20260811 00:13:18.864310 20294 raft_consensus.cc:686] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [term 3 LEADER]: Becoming Leader. State: Replica: a124e8d010f24686a8793e2134a7febc, State: Running, Role: LEADER
I20260811 00:13:18.864746 20294 consensus_queue.cc:237] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false } }
I20260811 00:13:18.866439 20297 sys_catalog.cc:455] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a124e8d010f24686a8793e2134a7febc. Latest consensus state: current_term: 3 leader_uuid: "a124e8d010f24686a8793e2134a7febc" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false } } }
I20260811 00:13:18.866777 20297 sys_catalog.cc:458] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:18.867457 20299 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:13:18.871634 20299 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:13:18.872599 20299 catalog_manager.cc:1259] Loaded cluster ID: 98709dcf80214cd5a07efe330d81c3d5
I20260811 00:13:18.872712 20299 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:13:18.874194 20299 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:13:18.875070 20299 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: Loaded TSK: 0
I20260811 00:13:18.876083 20299 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:13:18.925446 19316 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:13:18.932308 19316 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:13:18.936623 20295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.936149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:33332)
0811 00:13:18.936351 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:18.936454 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 00:13:18.937047 18986 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [term 3 FOLLOWER]: Refusing update from remote peer a124e8d010f24686a8793e2134a7febc: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 00:13:18.937706 20295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.937336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:33342)
0811 00:13:18.937532 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:18.937597 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 00:13:18.937870 19298 consensus_peers.cc:487] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc -> Peer aad6ba1a2f694580b8fa3dc892b1d83b (127.7.94.62:36587): Couldn't send request to peer aad6ba1a2f694580b8fa3dc892b1d83b. Status: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 00:13:18.937906 20294 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 00:13:18.940315 20303 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a124e8d010f24686a8793e2134a7febc. Latest consensus state: current_term: 3 leader_uuid: "a124e8d010f24686a8793e2134a7febc" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false } } }
I20260811 00:13:18.940558 20303 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:13:18.942368 20294 sys_catalog.cc:455] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "a124e8d010f24686a8793e2134a7febc" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false } } }
I20260811 00:13:18.942699 20294 sys_catalog.cc:458] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:18.946116 20303 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "a124e8d010f24686a8793e2134a7febc" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "aad6ba1a2f694580b8fa3dc892b1d83b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36587 } } peers { permanent_uuid: "f2fdaa9fb639489a89d8ff8c2bb0dfb4" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 37589 } attrs { promote: false } } peers { permanent_uuid: "a124e8d010f24686a8793e2134a7febc" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 39421 } attrs { promote: false } } }
I20260811 00:13:18.946419 20303 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2fdaa9fb639489a89d8ff8c2bb0dfb4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:13:18.967226 20317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.952032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:35337 (local address 127.0.0.1:52628)
0811 00:13:18.953000 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:18.953024 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:18.953157 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:18.961435 (+ 8278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:18.961440 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:18.961466 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:18.962014 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:18.962028 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:18.963438 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:18.963444 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:18.964966 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:18.964978 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:18.966622 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:18.966629 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:18.966768 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:18.966792 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 00:13:18.966849 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 00:13:18.966942 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"thread_start_us":10307,"threads_started":1}
I20260811 00:13:18.968204 20320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.952552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52628 (local address 127.7.94.2:35337)
0811 00:13:18.961058 (+ 8506us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:18.961065 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:18.961083 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:13:18.961159 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:18.961165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:18.961274 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:18.962009 (+ 735us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:18.962191 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:18.963300 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:18.965132 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:18.966395 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:18.967613 (+ 1218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:18.967741 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:18.967793 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 00:13:18.967928 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8366,"thread_start_us":8256,"threads_started":1}
I20260811 00:13:18.968998 19766 tablet_service.cc:1467] Processing CreateTablet for tablet fd2496f3c93e4a36921baec9d81f6c52 (DEFAULT_TABLE table=test-table [id=f4d6a301d7a941c9b6ccc0f9a2fa8283]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:18.969522 20315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.952375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:34285 (local address 127.0.0.1:43108)
0811 00:13:18.962596 (+ 10221us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:18.962614 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:18.962744 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:18.963170 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:18.963174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:18.963198 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:18.963612 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:18.963632 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:18.965080 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:18.965087 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:18.967393 (+ 2306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:18.967409 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:18.968740 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:18.968747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:18.968924 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:18.968955 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 00:13:18.969044 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 00:13:18.969178 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10105,"thread_start_us":385,"threads_started":1}
I20260811 00:13:18.969555 19766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd2496f3c93e4a36921baec9d81f6c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:18.969920 20295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.951498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:36059 (local address 127.0.0.1:55942)
0811 00:13:18.951868 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:18.951887 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:18.952019 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:18.963465 (+ 11446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:18.963473 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:18.963508 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:18.963963 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:18.963977 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:18.965468 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:18.965474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:18.967579 (+ 2105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:18.967591 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:18.969170 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:18.969178 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:18.969395 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:18.969427 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 00:13:18.969528 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 00:13:18.969664 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267}
I20260811 00:13:18.970168 20313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.951904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55942 (local address 127.7.94.1:36059)
0811 00:13:18.963156 (+ 11252us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:18.963162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:18.963179 (+ 17us) server_negotiation.cc:407] Connection header received
0811 00:13:18.963242 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:18.963248 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:18.963317 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:18.964127 (+ 810us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:18.964294 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:18.965308 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:18.967735 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:18.968945 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:18.969080 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:18.969179 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:18.969596 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 00:13:18.969888 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11129,"thread_start_us":11030,"threads_started":1}
I20260811 00:13:18.970207 20321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.952736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43108 (local address 127.7.94.3:34285)
0811 00:13:18.961934 (+ 9198us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:18.961942 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:18.962727 (+ 785us) server_negotiation.cc:407] Connection header received
0811 00:13:18.962880 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:18.962888 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:18.962975 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:18.963134 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:18.963827 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:18.964943 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:18.967598 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:18.968537 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:18.968688 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:18.968794 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:18.969768 (+ 974us) server_negotiation.cc:299] Negotiation successful
0811 00:13:18.969903 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9032,"thread_start_us":8920,"threads_started":1}
I20260811 00:13:18.970947 19522 tablet_service.cc:1467] Processing CreateTablet for tablet fd2496f3c93e4a36921baec9d81f6c52 (DEFAULT_TABLE table=test-table [id=f4d6a301d7a941c9b6ccc0f9a2fa8283]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:18.970953 20017 tablet_service.cc:1467] Processing CreateTablet for tablet fd2496f3c93e4a36921baec9d81f6c52 (DEFAULT_TABLE table=test-table [id=f4d6a301d7a941c9b6ccc0f9a2fa8283]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:18.971381 19522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd2496f3c93e4a36921baec9d81f6c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:18.971400 20017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd2496f3c93e4a36921baec9d81f6c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:18.976697 20322 tablet_bootstrap.cc:492] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c: Bootstrap starting.
I20260811 00:13:18.977592 20324 tablet_bootstrap.cc:492] T fd2496f3c93e4a36921baec9d81f6c52 P c488dd19f66d45419d4054c024614935: Bootstrap starting.
I20260811 00:13:18.978744 20327 tablet_bootstrap.cc:492] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8: Bootstrap starting.
I20260811 00:13:18.979032 20322 tablet_bootstrap.cc:654] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:18.979493 20324 tablet_bootstrap.cc:654] T fd2496f3c93e4a36921baec9d81f6c52 P c488dd19f66d45419d4054c024614935: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:18.980795 20327 tablet_bootstrap.cc:654] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:18.981391 20322 tablet_bootstrap.cc:492] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c: No bootstrap required, opened a new log
I20260811 00:13:18.981560 20322 ts_tablet_manager.cc:1397] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 00:13:18.981729 20324 tablet_bootstrap.cc:492] T fd2496f3c93e4a36921baec9d81f6c52 P c488dd19f66d45419d4054c024614935: No bootstrap required, opened a new log
I20260811 00:13:18.981870 20324 ts_tablet_manager.cc:1397] T fd2496f3c93e4a36921baec9d81f6c52 P c488dd19f66d45419d4054c024614935: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 00:13:18.982384 20322 raft_consensus.cc:348] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 } } peers { permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 } } peers { permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 } }
I20260811 00:13:18.982672 20322 raft_consensus.cc:374] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:18.982771 20322 raft_consensus.cc:729] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6905d72f434b7f86d5d92f6a46184c, State: Initialized, Role: FOLLOWER
I20260811 00:13:18.982652 20324 raft_consensus.cc:348] T fd2496f3c93e4a36921baec9d81f6c52 P c488dd19f66d45419d4054c024614935 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 } } peers { permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 } } peers { permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 } }
I20260811 00:13:18.982887 20324 raft_consensus.cc:374] T fd2496f3c93e4a36921baec9d81f6c52 P c488dd19f66d45419d4054c024614935 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:18.982894 20327 tablet_bootstrap.cc:492] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8: No bootstrap required, opened a new log
I20260811 00:13:18.982996 20324 raft_consensus.cc:729] T fd2496f3c93e4a36921baec9d81f6c52 P c488dd19f66d45419d4054c024614935 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c488dd19f66d45419d4054c024614935, State: Initialized, Role: FOLLOWER
I20260811 00:13:18.983040 20327 ts_tablet_manager.cc:1397] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260811 00:13:18.983047 20322 consensus_queue.cc:260] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 } } peers { permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 } } peers { permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 } }
I20260811 00:13:18.983234 20324 consensus_queue.cc:260] T fd2496f3c93e4a36921baec9d81f6c52 P c488dd19f66d45419d4054c024614935 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 } } peers { permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 } } peers { permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 } }
I20260811 00:13:18.983947 20327 raft_consensus.cc:348] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 } } peers { permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 } } peers { permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 } }
I20260811 00:13:18.984243 20327 raft_consensus.cc:374] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:18.984360 20322 ts_tablet_manager.cc:1428] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 00:13:18.984360 20327 raft_consensus.cc:729] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 707ae6eda37543b087110d357cceadb8, State: Initialized, Role: FOLLOWER
I20260811 00:13:18.984640 20324 ts_tablet_manager.cc:1428] T fd2496f3c93e4a36921baec9d81f6c52 P c488dd19f66d45419d4054c024614935: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 00:13:18.984612 20327 consensus_queue.cc:260] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 } } peers { permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 } } peers { permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 } }
I20260811 00:13:18.987130 20327 ts_tablet_manager.cc:1428] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260811 00:13:18.989327 20132 heartbeater.cc:503] Master 127.7.94.60:39421 was elected leader, sending a full tablet report...
I20260811 00:13:18.991083 20341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.985792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.3:52009)
0811 00:13:18.990580 (+ 4788us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:18.990719 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4636,"thread_start_us":4490,"threads_started":1}
I20260811 00:13:18.991633 19633 heartbeater.cc:503] Master 127.7.94.60:39421 was elected leader, sending a full tablet report...
I20260811 00:13:18.993422 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.991663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.3:58345)
0811 00:13:18.992239 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:18.992946 (+ 707us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":291,"threads_started":1}
I20260811 00:13:18.994489 19878 heartbeater.cc:503] Master 127.7.94.60:39421 was elected leader, sending a full tablet report...
I20260811 00:13:18.996884 20340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.985085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.1:49919)
0811 00:13:18.996549 (+ 11464us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:18.996671 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11349,"threads_started":1}
I20260811 00:13:18.998271 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.993154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.2:57289)
0811 00:13:18.997875 (+ 4721us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:18.997999 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4578,"thread_start_us":4323,"threads_started":1}
I20260811 00:13:18.999310 20340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.998743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.1:42885)
0811 00:13:18.999062 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:18.999159 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226}
W20260811 00:13:18.999796 19887 tablet.cc:2369] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:13:19.000743 20339 raft_consensus.cc:482] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:13:19.001092 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:18.999131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.2:57779)
0811 00:13:19.000750 (+ 1619us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:19.000948 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1508}
I20260811 00:13:19.000981 20339 raft_consensus.cc:504] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 } } peers { permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 } } peers { permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 } }
I20260811 00:13:19.002712 20339 leader_election.cc:288] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c488dd19f66d45419d4054c024614935 (127.7.94.1:36059), 6c6905d72f434b7f86d5d92f6a46184c (127.7.94.2:35337)
I20260811 00:13:19.008395 20320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:19.003133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:34439 (local address 127.7.94.2:35337)
0811 00:13:19.003294 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:19.003299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:19.003315 (+ 16us) server_negotiation.cc:407] Connection header received
0811 00:13:19.003371 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:19.003374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:19.003424 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:19.003546 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:19.004138 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:19.004812 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:19.006727 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:19.007485 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:19.007640 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:19.007739 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:19.008044 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 00:13:19.008173 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 00:13:19.008446 20341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:19.002841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:36059 (local address 127.7.94.3:41339)
0811 00:13:19.003005 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:19.003025 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:19.003153 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:19.003504 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:19.003512 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:19.003540 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:19.003895 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:19.003910 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:19.004925 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:19.004931 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:19.006557 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:19.006570 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:19.007686 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:19.007692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:19.007871 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:19.007919 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 00:13:19.008027 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 00:13:19.008168 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":112}
I20260811 00:13:19.008440 20313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:19.002922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:41339 (local address 127.7.94.1:36059)
0811 00:13:19.003213 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:19.003219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:19.003237 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:13:19.003300 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:19.003307 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:19.003376 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:19.003454 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:19.004059 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:19.004788 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:19.006721 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:19.007478 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:19.007603 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:19.007719 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:19.008072 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 00:13:19.008196 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260811 00:13:19.008842 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:19.002938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:35337 (local address 127.7.94.3:34439)
0811 00:13:19.003111 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:19.003131 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:19.003251 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:19.003617 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:19.003624 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:19.003658 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:19.003968 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:19.004000 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:19.004960 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:19.004963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:19.006558 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:19.006571 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:19.007706 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:19.007712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:19.007874 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:19.007914 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 00:13:19.007985 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 00:13:19.008159 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":110}
I20260811 00:13:19.009351 19553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd2496f3c93e4a36921baec9d81f6c52" candidate_uuid: "707ae6eda37543b087110d357cceadb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c488dd19f66d45419d4054c024614935" is_pre_election: true
I20260811 00:13:19.009470 19805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd2496f3c93e4a36921baec9d81f6c52" candidate_uuid: "707ae6eda37543b087110d357cceadb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6905d72f434b7f86d5d92f6a46184c" is_pre_election: true
I20260811 00:13:19.009690 19805 raft_consensus.cc:2454] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 707ae6eda37543b087110d357cceadb8 in term 0.
I20260811 00:13:19.009696 19553 raft_consensus.cc:2454] T fd2496f3c93e4a36921baec9d81f6c52 P c488dd19f66d45419d4054c024614935 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 707ae6eda37543b087110d357cceadb8 in term 0.
I20260811 00:13:19.010239 19932 leader_election.cc:302] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6c6905d72f434b7f86d5d92f6a46184c, 707ae6eda37543b087110d357cceadb8, c488dd19f66d45419d4054c024614935; no voters:
I20260811 00:13:19.010762 20339 raft_consensus.cc:2781] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:13:19.010927 20339 raft_consensus.cc:482] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:13:19.011099 20339 raft_consensus.cc:3037] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:19.013692 20339 raft_consensus.cc:504] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 } } peers { permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 } } peers { permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 } }
I20260811 00:13:19.014343 20339 leader_election.cc:288] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [CANDIDATE]: Term 1 election: Requested vote from peers c488dd19f66d45419d4054c024614935 (127.7.94.1:36059), 6c6905d72f434b7f86d5d92f6a46184c (127.7.94.2:35337)
I20260811 00:13:19.014824 19553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd2496f3c93e4a36921baec9d81f6c52" candidate_uuid: "707ae6eda37543b087110d357cceadb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c488dd19f66d45419d4054c024614935"
I20260811 00:13:19.014842 19805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd2496f3c93e4a36921baec9d81f6c52" candidate_uuid: "707ae6eda37543b087110d357cceadb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c6905d72f434b7f86d5d92f6a46184c"
I20260811 00:13:19.015003 19805 raft_consensus.cc:3037] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:19.015007 19553 raft_consensus.cc:3037] T fd2496f3c93e4a36921baec9d81f6c52 P c488dd19f66d45419d4054c024614935 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:19.016947 19805 raft_consensus.cc:2454] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 707ae6eda37543b087110d357cceadb8 in term 1.
I20260811 00:13:19.017028 19553 raft_consensus.cc:2454] T fd2496f3c93e4a36921baec9d81f6c52 P c488dd19f66d45419d4054c024614935 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 707ae6eda37543b087110d357cceadb8 in term 1.
I20260811 00:13:19.017345 19932 leader_election.cc:302] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c6905d72f434b7f86d5d92f6a46184c, 707ae6eda37543b087110d357cceadb8; no voters:
I20260811 00:13:19.017647 20339 raft_consensus.cc:2781] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:13:19.018100 20339 raft_consensus.cc:686] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [term 1 LEADER]: Becoming Leader. State: Replica: 707ae6eda37543b087110d357cceadb8, State: Running, Role: LEADER
I20260811 00:13:19.018685 20339 consensus_queue.cc:237] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 } } peers { permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 } } peers { permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 } }
I20260811 00:13:19.019074 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:19.018624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.3:58315)
0811 00:13:19.018782 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:19.018868 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 00:13:19.022810 19317 catalog_manager.cc:5495] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 707ae6eda37543b087110d357cceadb8 (127.7.94.3). New cstate: current_term: 1 leader_uuid: "707ae6eda37543b087110d357cceadb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "707ae6eda37543b087110d357cceadb8" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 34285 } health_report { overall_health: HEALTHY } } }
I20260811 00:13:19.038973 20321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:19.034132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43124 (local address 127.7.94.3:34285)
0811 00:13:19.034309 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:19.034313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:19.034329 (+ 16us) server_negotiation.cc:407] Connection header received
0811 00:13:19.034452 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:19.034456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:19.034526 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:19.034654 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:13:19.035109 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:19.035815 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:19.037023 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:19.037488 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:19.037581 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:19.037953 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:13:19.038499 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:13:19.038613 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:19.038689 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 00:13:19.038770 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 00:13:19.043116 19805 raft_consensus.cc:1261] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c [term 1 FOLLOWER]: Refusing update from remote peer 707ae6eda37543b087110d357cceadb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:13:19.043506 19553 raft_consensus.cc:1261] T fd2496f3c93e4a36921baec9d81f6c52 P c488dd19f66d45419d4054c024614935 [term 1 FOLLOWER]: Refusing update from remote peer 707ae6eda37543b087110d357cceadb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:13:19.043903 20347 consensus_queue.cc:1035] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c6905d72f434b7f86d5d92f6a46184c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:13:19.044356 20339 consensus_queue.cc:1035] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c488dd19f66d45419d4054c024614935" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:13:19.047338 20340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:19.046749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.1:37313)
0811 00:13:19.046939 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:19.047048 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 00:13:19.047580 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:19.046742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.2:41163)
0811 00:13:19.047199 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:19.047335 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295}
I20260811 00:13:19.054629 20353 mvcc.cc:205] Tried to move back new op lower bound from 7317123887272808448 to 7317123887185321984. Current Snapshot: MvccSnapshot[applied={T|T < 7317123887272808448}]
I20260811 00:13:19.056569 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:19.056092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.3:52067)
0811 00:13:19.056265 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:19.056357 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 00:13:19.056569 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:19.056096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.2:48217)
0811 00:13:19.056342 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:19.056420 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260811 00:13:19.060719 20340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:19.060272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.7.94.1:59557)
0811 00:13:19.060450 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:19.060556 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 00:13:19.071306 19316 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:13:19.071655 19316 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:13:19.077998 19316 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a124e8d010f24686a8793e2134a7febc: Sending DeleteTablet for 3 replicas of tablet fd2496f3c93e4a36921baec9d81f6c52
I20260811 00:13:19.079162 19766 tablet_service.cc:1514] Processing DeleteTablet for tablet fd2496f3c93e4a36921baec9d81f6c52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:19 UTC) from {username='oryx', principal='oryx/127.7.94.60@KRBTEST.COM'} at 127.0.0.1:52628
I20260811 00:13:19.079214 19522 tablet_service.cc:1514] Processing DeleteTablet for tablet fd2496f3c93e4a36921baec9d81f6c52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:19 UTC) from {username='oryx', principal='oryx/127.7.94.60@KRBTEST.COM'} at 127.0.0.1:55942
I20260811 00:13:19.079982 20365 tablet_replica.cc:330] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c: stopping tablet replica
I20260811 00:13:19.080363 20365 raft_consensus.cc:2229] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:13:19.080382 20017 tablet_service.cc:1514] Processing DeleteTablet for tablet fd2496f3c93e4a36921baec9d81f6c52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:19 UTC) from {username='oryx', principal='oryx/127.7.94.60@KRBTEST.COM'} at 127.0.0.1:43108
I20260811 00:13:19.080899 20365 raft_consensus.cc:2258] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:19.082659 20365 ts_tablet_manager.cc:1905] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:19.085915 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 19272
I20260811 00:13:19.092227 20367 tablet_replica.cc:330] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8: stopping tablet replica
I20260811 00:13:19.092514 20367 raft_consensus.cc:2229] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:13:19.093374 20367 raft_consensus.cc:2258] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:19.094017 20366 tablet_replica.cc:330] T fd2496f3c93e4a36921baec9d81f6c52 P c488dd19f66d45419d4054c024614935: stopping tablet replica
I20260811 00:13:19.094261 20366 raft_consensus.cc:2229] T fd2496f3c93e4a36921baec9d81f6c52 P c488dd19f66d45419d4054c024614935 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:13:19.094588 20366 raft_consensus.cc:2258] T fd2496f3c93e4a36921baec9d81f6c52 P c488dd19f66d45419d4054c024614935 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:19.094607 20367 ts_tablet_manager.cc:1905] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:19.097568 20366 ts_tablet_manager.cc:1905] T fd2496f3c93e4a36921baec9d81f6c52 P c488dd19f66d45419d4054c024614935: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:19.098591 20365 ts_tablet_manager.cc:1918] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:13:19.099502 20365 log.cc:1196] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/fd2496f3c93e4a36921baec9d81f6c52
I20260811 00:13:19.102285 20365 ts_tablet_manager.cc:1939] T fd2496f3c93e4a36921baec9d81f6c52 P 6c6905d72f434b7f86d5d92f6a46184c: Deleting consensus metadata
I20260811 00:13:19.103390 20367 ts_tablet_manager.cc:1918] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:13:19.103557 20367 log.cc:1196] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/fd2496f3c93e4a36921baec9d81f6c52
I20260811 00:13:19.103806 19296 catalog_manager.cc:4841] TS 6c6905d72f434b7f86d5d92f6a46184c (127.7.94.2:35337): tablet fd2496f3c93e4a36921baec9d81f6c52 (table test-table [id=f4d6a301d7a941c9b6ccc0f9a2fa8283]) successfully deleted
I20260811 00:13:19.104146 20367 ts_tablet_manager.cc:1939] T fd2496f3c93e4a36921baec9d81f6c52 P 707ae6eda37543b087110d357cceadb8: Deleting consensus metadata
I20260811 00:13:19.105352 19293 catalog_manager.cc:4841] TS 707ae6eda37543b087110d357cceadb8 (127.7.94.3:34285): tablet fd2496f3c93e4a36921baec9d81f6c52 (table test-table [id=f4d6a301d7a941c9b6ccc0f9a2fa8283]) successfully deleted
I20260811 00:13:19.115290 20366 ts_tablet_manager.cc:1918] T fd2496f3c93e4a36921baec9d81f6c52 P c488dd19f66d45419d4054c024614935: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:13:19.115443 20366 log.cc:1196] T fd2496f3c93e4a36921baec9d81f6c52 P c488dd19f66d45419d4054c024614935: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/fd2496f3c93e4a36921baec9d81f6c52
I20260811 00:13:19.115986 20366 ts_tablet_manager.cc:1939] T fd2496f3c93e4a36921baec9d81f6c52 P c488dd19f66d45419d4054c024614935: Deleting consensus metadata
I20260811 00:13:19.117260 19295 catalog_manager.cc:4841] TS c488dd19f66d45419d4054c024614935 (127.7.94.1:36059): tablet fd2496f3c93e4a36921baec9d81f6c52 (table test-table [id=f4d6a301d7a941c9b6ccc0f9a2fa8283]) successfully deleted
I20260811 00:13:19.121284 20313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:19.086811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55954 (local address 127.7.94.1:36059)
0811 00:13:19.087087 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:19.087094 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:19.099172 (+ 12078us) server_negotiation.cc:407] Connection header received
0811 00:13:19.099756 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:19.099764 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:19.099864 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:19.099959 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:19.104742 (+ 4783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:19.105894 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:19.107785 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:19.108390 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:19.108487 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:19.112265 (+ 3778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:19.112284 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:19.112304 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:19.112350 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:19.117151 (+ 4801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:19.117932 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:19.117942 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:19.117945 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:19.118128 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:19.118619 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:19.118632 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:19.118635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:19.119129 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:19.119433 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:19.120618 (+ 1185us) server_negotiation.cc:299] Negotiation successful
0811 00:13:19.120856 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260811 00:13:19.122103 19622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:19.266472 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 19272
I20260811 00:13:19.306979 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 19652
I20260811 00:13:19.322564 20320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:19.307830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52632 (local address 127.7.94.2:35337)
0811 00:13:19.308014 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:19.308018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:19.308040 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:13:19.308283 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:19.308287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:19.308350 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:19.308429 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:19.310228 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:19.310984 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:19.312849 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:19.313353 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:19.313449 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:19.316766 (+ 3317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:19.316781 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:19.316797 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:19.316833 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:19.319832 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:19.320581 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:19.320588 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:19.320590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:19.320761 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:19.321131 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:19.321134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:19.321136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:19.321540 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:19.321772 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:19.322081 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 00:13:19.322252 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 00:13:19.323324 19867 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:19.354763 20295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:19.354266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:33358)
0811 00:13:19.354468 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:19.354574 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 00:13:19.355926 20295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:19.355496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:36587 (local address 127.0.0.1:33368)
0811 00:13:19.355655 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:19.355744 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 00:13:19.418825 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 19652
I20260811 00:13:19.453460 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 19902
I20260811 00:13:19.469617 20321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:19.454332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43138 (local address 127.7.94.3:34285)
0811 00:13:19.454507 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:19.454511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:19.454532 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:13:19.454726 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:19.454731 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:19.454808 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:19.454889 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:19.456703 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:19.457586 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:19.459350 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:19.459780 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:19.459871 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:19.463309 (+ 3438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:19.463323 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:19.463339 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:19.463374 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:19.466576 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:19.467442 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:19.467449 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:19.467451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:19.467619 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:19.468025 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:19.468031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:19.468034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:19.468410 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:19.468775 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:19.469036 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 00:13:19.469289 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 00:13:19.470333 20125 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:19.601503 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 19902
I20260811 00:13:19.633611 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 18747
I20260811 00:13:19.634778 19022 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:19.752861 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 18747
I20260811 00:13:19.779678 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 18913
I20260811 00:13:19.780784 19384 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:19.904146 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 18913
2026-08-11T00:13:19Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11862 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:13:20 dist-test-slave-8hhm krb5kdc[20383](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:13:20 dist-test-slave-8hhm krb5kdc[20383](info): set up 2 sockets
Aug 11 00:13:20 dist-test-slave-8hhm krb5kdc[20383](info): commencing operation
krb5kdc: starting...
W20260811 00:13:21.263003 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:13:21 dist-test-slave-8hhm krb5kdc[20383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:13:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:13:21Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.62.keytab.
Entry for principal oryx/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.62.keytab.
I20260811 00:13:21.528440 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:43779
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.62.keytab
--principal=oryx/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:46089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:43779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:21.861431 20407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:21.861779 20407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:21.861951 20407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:21.869782 20407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:13:21.869915 20407 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 00:13:21.869990 20407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:21.870105 20407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:21.870200 20407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:13:21.870288 20407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:13:21.880584 20407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:46089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:43779
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.62.keytab
--principal=oryx/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:43779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:21.882184 20407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:21.883607 20407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:21.895715 20416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:21.895646 20420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:21.895695 20418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:21.896486 20407 server_base.cc:1034] running on GCE node
I20260811 00:13:21.897398 20407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:21.898811 20407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:21.900066 20407 hybrid_clock.cc:648] HybridClock initialized: now 1786407201900011 us; error 29 us; skew 500 ppm
I20260811 00:13:21.900403 20407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:21 dist-test-slave-8hhm krb5kdc[20383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407201, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:21.907226 20407 init.cc:377] Logged in from keytab as oryx/127.7.94.62@KRBTEST.COM (short username oryx)
I20260811 00:13:21.908874 20407 webserver.cc:460] Webserver started at http://127.7.94.62:43637/ using document root <none> and password file <none>
I20260811 00:13:21.909953 20407 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:21.910084 20407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:21.910414 20407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:21.913690 20407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "a2b21918e3524c69a10ca87835f94a2f"
format_stamp: "Formatted at 2026-08-11 00:13:21 on dist-test-slave-8hhm"
server_key: "12b9441dada1f2658e723eabf4376f27"
server_key_iv: "077d23ee16aa3c94c90715f12a38213a"
server_key_version: "encryptionkey@0"
I20260811 00:13:21.914636 20407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "a2b21918e3524c69a10ca87835f94a2f"
format_stamp: "Formatted at 2026-08-11 00:13:21 on dist-test-slave-8hhm"
server_key: "12b9441dada1f2658e723eabf4376f27"
server_key_iv: "077d23ee16aa3c94c90715f12a38213a"
server_key_version: "encryptionkey@0"
I20260811 00:13:21.919917 20407 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 00:13:21.923471 20427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:21.924875 20407 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 00:13:21.925087 20407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "a2b21918e3524c69a10ca87835f94a2f"
format_stamp: "Formatted at 2026-08-11 00:13:21 on dist-test-slave-8hhm"
server_key: "12b9441dada1f2658e723eabf4376f27"
server_key_iv: "077d23ee16aa3c94c90715f12a38213a"
server_key_version: "encryptionkey@0"
I20260811 00:13:21.925360 20407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:21.961021 20407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:21.965160 20407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:21.965478 20407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:21.987740 20407 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:43779
I20260811 00:13:21.987766 20540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:43779 every 8 connection(s)
I20260811 00:13:21.989179 20407 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:13:21.993235 20542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:21.995589 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 20407
I20260811 00:13:21.995831 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:13:21.996737 7544 external_mini_cluster.cc:1428] Setting key 38936e37878bd84fa4581481de1d450d
I20260811 00:13:22.007278 20542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f: Bootstrap starting.
I20260811 00:13:22.011827 20542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f: Neither blocks nor log segments found. Creating new log.
Aug 11 00:13:22 dist-test-slave-8hhm krb5kdc[20383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM
I20260811 00:13:22.013258 20542 log.cc:824] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:13:22.016376 20542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f: No bootstrap required, opened a new log
I20260811 00:13:22.021581 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:21.999018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46898 (local address 127.7.94.62:43779)
0811 00:13:21.999892 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:21.999904 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:21.999947 (+ 43us) server_negotiation.cc:407] Connection header received
0811 00:13:22.001759 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:22.001782 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:22.002162 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:22.002467 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:22.004442 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:22.005825 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:22.007639 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:22.008806 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:22.008947 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:22.014137 (+ 5190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:22.014148 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:22.014162 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:22.014197 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:22.017515 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:22.018190 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:22.018196 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:22.018202 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:22.018370 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:22.018771 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:22.018785 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:22.018788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:22.019314 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:22.019628 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:22.020104 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 00:13:22.020504 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":464,"threads_started":1}
I20260811 00:13:22.025046 20542 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2b21918e3524c69a10ca87835f94a2f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43779 } }
I20260811 00:13:22.025504 20542 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:22.025621 20542 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2b21918e3524c69a10ca87835f94a2f, State: Initialized, Role: FOLLOWER
I20260811 00:13:22.026235 20542 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2b21918e3524c69a10ca87835f94a2f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43779 } }
I20260811 00:13:22.026532 20542 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:13:22.026695 20542 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:13:22.026892 20542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:22.029672 20542 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2b21918e3524c69a10ca87835f94a2f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43779 } }
I20260811 00:13:22.030318 20542 leader_election.cc:302] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2b21918e3524c69a10ca87835f94a2f; no voters:
I20260811 00:13:22.030866 20542 leader_election.cc:288] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:13:22.030967 20548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:13:22.031752 20548 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f [term 1 LEADER]: Becoming Leader. State: Replica: a2b21918e3524c69a10ca87835f94a2f, State: Running, Role: LEADER
I20260811 00:13:22.032349 20548 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2b21918e3524c69a10ca87835f94a2f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43779 } }
I20260811 00:13:22.033402 20542 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:13:22.035411 20550 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2b21918e3524c69a10ca87835f94a2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2b21918e3524c69a10ca87835f94a2f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43779 } } }
I20260811 00:13:22.035423 20551 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2b21918e3524c69a10ca87835f94a2f. Latest consensus state: current_term: 1 leader_uuid: "a2b21918e3524c69a10ca87835f94a2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2b21918e3524c69a10ca87835f94a2f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43779 } } }
I20260811 00:13:22.035851 20550 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:22.035974 20551 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:22.037683 20559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:13:22.042554 20559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:13:22.051556 20559 catalog_manager.cc:1347] Generated new cluster ID: d0cdf1db99274f27890c9f5605b9a328
I20260811 00:13:22.051692 20559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:13:22.062671 20559 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:13:22.063982 20559 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:13:22.078751 20559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f: Generated new TSK 0
I20260811 00:13:22.079591 20559 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.1.keytab.
Entry for principal oryx/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.1.keytab.
I20260811 00:13:22.145653 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:43779
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.1.keytab
--principal=oryx/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:46089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:22.484730 20584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:22.485069 20584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:22.485272 20584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:22.492268 20584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:22.492417 20584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:22.492579 20584 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:13:22.492727 20584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:13:22.502125 20584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:46089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.1.keytab
--principal=oryx/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.94.62:43779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:22.503383 20584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:22.504698 20584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:22.517755 20591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:22.517830 20593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:22.517776 20595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:22.519145 20584 server_base.cc:1034] running on GCE node
I20260811 00:13:22.519768 20584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:22.520774 20584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:22.522012 20584 hybrid_clock.cc:648] HybridClock initialized: now 1786407202521950 us; error 40 us; skew 500 ppm
I20260811 00:13:22.522310 20584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:22 dist-test-slave-8hhm krb5kdc[20383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407202, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:22.525615 20584 init.cc:377] Logged in from keytab as oryx/127.7.94.1@KRBTEST.COM (short username oryx)
I20260811 00:13:22.527187 20584 webserver.cc:460] Webserver started at http://127.7.94.1:39295/ using document root <none> and password file <none>
I20260811 00:13:22.527870 20584 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:22.527980 20584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:22.528322 20584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:22.531241 20584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba"
format_stamp: "Formatted at 2026-08-11 00:13:22 on dist-test-slave-8hhm"
server_key: "73175b0ce5ff717b946b013d41f777c1"
server_key_iv: "935903995ca74e7fee19c09a37c789e8"
server_key_version: "encryptionkey@0"
I20260811 00:13:22.532135 20584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba"
format_stamp: "Formatted at 2026-08-11 00:13:22 on dist-test-slave-8hhm"
server_key: "73175b0ce5ff717b946b013d41f777c1"
server_key_iv: "935903995ca74e7fee19c09a37c789e8"
server_key_version: "encryptionkey@0"
I20260811 00:13:22.537597 20584 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 00:13:22.540906 20613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:22.542484 20584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:13:22.542714 20584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba"
format_stamp: "Formatted at 2026-08-11 00:13:22 on dist-test-slave-8hhm"
server_key: "73175b0ce5ff717b946b013d41f777c1"
server_key_iv: "935903995ca74e7fee19c09a37c789e8"
server_key_version: "encryptionkey@0"
I20260811 00:13:22.542974 20584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:22.563309 20584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:22.567442 20584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:22.567744 20584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:22.570828 20584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:22.570981 20584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:22.571139 20584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:22.571234 20584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:13:22 dist-test-slave-8hhm krb5kdc[20383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407202, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM
I20260811 00:13:22.593389 20626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:22.571059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:52277 (local address 127.7.94.62:43779)
0811 00:13:22.571583 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:22.571589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:22.572864 (+ 1275us) server_negotiation.cc:407] Connection header received
0811 00:13:22.575024 (+ 2160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:22.575035 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:22.575156 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:22.575303 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:22.577185 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:22.578620 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:22.580938 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:22.581554 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:22.581719 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:22.586692 (+ 4973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:22.586697 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:22.586700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:22.586735 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:22.590094 (+ 3359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:22.590932 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:22.590939 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:22.590943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:22.591078 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:22.591542 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:22.591549 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:22.591553 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:22.591859 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:22.592011 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:22.592736 (+ 725us) server_negotiation.cc:299] Negotiation successful
0811 00:13:22.592962 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":274,"threads_started":1}
I20260811 00:13:22.594867 20628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:22.571756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:43779 (local address 127.7.94.1:52277)
0811 00:13:22.572666 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:22.572715 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:22.574713 (+ 1998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:22.575500 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:22.575508 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:22.576037 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:22.576952 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:22.576984 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:22.578808 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:22.578823 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:22.580764 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:22.580783 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:22.581864 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:22.581876 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:22.582078 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:22.582979 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:22.583002 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:22.586369 (+ 3367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:22.590278 (+ 3909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:22.590285 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:22.590298 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:22.590796 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:22.591210 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:22.591214 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:22.591218 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:22.591420 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:22.592003 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:22.592011 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:22.592367 (+ 356us) client_negotiation.cc:769] Sending connection context
0811 00:13:22.592659 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 00:13:22.593198 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":408,"threads_started":1}
I20260811 00:13:22.620249 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:22.602958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:43779 (local address 127.7.94.1:58303)
0811 00:13:22.603952 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:22.603982 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:22.604167 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:22.604534 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:22.604542 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:22.605369 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:22.605890 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:22.605908 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:22.606909 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:22.606917 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:22.609202 (+ 2285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:22.609246 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:22.610358 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:22.610365 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:22.610605 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:22.611392 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:22.611421 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:22.613138 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:22.617046 (+ 3908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:22.617051 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:22.617056 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:22.617793 (+ 737us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:22.618313 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:22.618322 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:22.618325 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:22.618579 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:22.619393 (+ 814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:22.619403 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:22.619573 (+ 170us) client_negotiation.cc:769] Sending connection context
0811 00:13:22.619689 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 00:13:22.619920 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":569,"threads_started":1}
I20260811 00:13:22.620816 20626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:22.602784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:58303 (local address 127.7.94.62:43779)
0811 00:13:22.602963 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:22.602969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:22.604092 (+ 1123us) server_negotiation.cc:407] Connection header received
0811 00:13:22.604302 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:22.604306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:22.604376 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:22.604450 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:22.606051 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:22.606739 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:22.609411 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:22.610120 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:22.610259 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:22.613465 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:22.613477 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:22.613485 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:22.613542 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:22.616904 (+ 3362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:22.617962 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:22.617969 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:22.617972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:22.618160 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:22.618756 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:22.618764 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:22.618771 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:22.619223 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:22.619406 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:22.619898 (+ 492us) server_negotiation.cc:299] Negotiation successful
0811 00:13:22.620219 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 00:13:22.631273 20584 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:37117
I20260811 00:13:22.631295 20825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:37117 every 8 connection(s)
I20260811 00:13:22.632649 20584 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:13:22.633798 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 20584
I20260811 00:13:22.634073 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:13:22.634935 7544 external_mini_cluster.cc:1428] Setting key 593d7126cfd55b51be412b176bdd5deb
I20260811 00:13:22.635495 20827 heartbeater.cc:347] Connected to a master server at 127.7.94.62:43779
I20260811 00:13:22.635735 20827 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:22.636439 20827 heartbeater.cc:511] Master 127.7.94.62:43779 requested a full tablet report, sending...
I20260811 00:13:22.638684 20458 ts_manager.cc:194] Registered new tserver with Master: f5b4b9cf1f114d799efc1e159dcfe7ba (127.7.94.1:37117)
I20260811 00:13:22.640203 20458 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:52277
WARNING: no policy specified for oryx/127.7.94.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.94.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.2.keytab.
Entry for principal oryx/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.2.keytab.
I20260811 00:13:22.692217 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:43779
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.2.keytab
--principal=oryx/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:46089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:23.029008 20833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:23.029333 20833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:23.029515 20833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:23.036841 20833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:23.037001 20833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:23.037165 20833 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:13:23.037320 20833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:13:23.046892 20833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:46089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.2.keytab
--principal=oryx/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.94.62:43779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:23.048223 20833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:23.049652 20833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:23.063473 20848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:23.063529 20850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:23.063551 20852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:23.064620 20833 server_base.cc:1034] running on GCE node
I20260811 00:13:23.065204 20833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:23.066149 20833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:23.067369 20833 hybrid_clock.cc:648] HybridClock initialized: now 1786407203067305 us; error 32 us; skew 500 ppm
I20260811 00:13:23.067672 20833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:23 dist-test-slave-8hhm krb5kdc[20383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407203, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:23.070864 20833 init.cc:377] Logged in from keytab as oryx/127.7.94.2@KRBTEST.COM (short username oryx)
I20260811 00:13:23.072373 20833 webserver.cc:460] Webserver started at http://127.7.94.2:36129/ using document root <none> and password file <none>
I20260811 00:13:23.073084 20833 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:23.073272 20833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:23.073626 20833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:23.076547 20833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "4cab97073ee041138f8cbbab9cb7ac02"
format_stamp: "Formatted at 2026-08-11 00:13:23 on dist-test-slave-8hhm"
server_key: "7837bc8b9926e98c742cf89f0028438f"
server_key_iv: "44a7daec861193351b44581da26c3e9c"
server_key_version: "encryptionkey@0"
I20260811 00:13:23.077435 20833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "4cab97073ee041138f8cbbab9cb7ac02"
format_stamp: "Formatted at 2026-08-11 00:13:23 on dist-test-slave-8hhm"
server_key: "7837bc8b9926e98c742cf89f0028438f"
server_key_iv: "44a7daec861193351b44581da26c3e9c"
server_key_version: "encryptionkey@0"
I20260811 00:13:23.082471 20833 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 00:13:23.085950 20867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:23.087263 20833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:13:23.087438 20833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "4cab97073ee041138f8cbbab9cb7ac02"
format_stamp: "Formatted at 2026-08-11 00:13:23 on dist-test-slave-8hhm"
server_key: "7837bc8b9926e98c742cf89f0028438f"
server_key_iv: "44a7daec861193351b44581da26c3e9c"
server_key_version: "encryptionkey@0"
I20260811 00:13:23.087674 20833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:23.103065 20833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:23.106989 20833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:23.107278 20833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:23.110102 20833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:23.110248 20833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:23.110450 20833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:23.110541 20833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:13:23 dist-test-slave-8hhm krb5kdc[20383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407203, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM
I20260811 00:13:23.136332 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:23.111202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:43779 (local address 127.7.94.2:50385)
0811 00:13:23.112041 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:23.112084 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:23.114274 (+ 2190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:23.115096 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:23.115105 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:23.115996 (+ 891us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:23.116891 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:23.116922 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.118485 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.118493 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:23.120792 (+ 2299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:23.120815 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.121997 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.122011 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:23.122331 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:23.123232 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:23.123266 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:23.126718 (+ 3452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:23.131122 (+ 4404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:23.131133 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:23.131155 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:23.131994 (+ 839us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:23.132410 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:23.132423 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:23.132425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:23.132772 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:23.133673 (+ 901us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:23.133680 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:23.134238 (+ 558us) client_negotiation.cc:769] Sending connection context
0811 00:13:23.134677 (+ 439us) client_negotiation.cc:240] Negotiation successful
0811 00:13:23.135270 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":383,"threads_started":1}
I20260811 00:13:23.137331 20626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:23.110508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:50385 (local address 127.7.94.62:43779)
0811 00:13:23.110698 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:23.110703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:23.112215 (+ 1512us) server_negotiation.cc:407] Connection header received
0811 00:13:23.114594 (+ 2379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:23.114600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:23.114729 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:23.114832 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:23.117136 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.118284 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.121001 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.121731 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.121894 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:23.126994 (+ 5100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:23.127003 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:23.127011 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:23.127051 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:23.130947 (+ 3896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:23.132136 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:23.132140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:23.132142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:23.132266 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:23.132948 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:23.132958 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:23.132960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:23.133431 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:23.133680 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:23.136852 (+ 3172us) server_negotiation.cc:299] Negotiation successful
0811 00:13:23.137019 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 00:13:23.160637 21011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:23.144042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:43779 (local address 127.7.94.2:38465)
0811 00:13:23.144794 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:23.144819 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:23.144992 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:23.145369 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:23.145374 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:23.146170 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:23.146675 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:23.146691 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.148093 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.148102 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:23.149995 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:23.150008 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.151179 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.151183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:23.151300 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:23.151989 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:23.152005 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:23.153080 (+ 1075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:23.157513 (+ 4433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:23.157521 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:23.157527 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:23.158113 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:23.158573 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:23.158590 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:23.158594 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:23.158794 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:23.159790 (+ 996us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:23.159794 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:23.159948 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 00:13:23.160061 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 00:13:23.160300 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":384,"threads_started":1}
I20260811 00:13:23.161021 20626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:23.144070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:38465 (local address 127.7.94.62:43779)
0811 00:13:23.144292 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:23.144300 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:23.144918 (+ 618us) server_negotiation.cc:407] Connection header received
0811 00:13:23.145121 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:23.145125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:23.145184 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:23.145342 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:23.146889 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.147943 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.150147 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.150745 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.150908 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:23.153339 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:23.153348 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:23.153356 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:23.153405 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:23.157325 (+ 3920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:23.158287 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:23.158292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:23.158295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:23.158452 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:23.158945 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:23.158972 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:23.158978 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:23.159598 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:23.159846 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:23.160221 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 00:13:23.160534 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:13:23.167141 21085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:43447 every 8 connection(s)
I20260811 00:13:23.167168 20833 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:43447
I20260811 00:13:23.168365 20833 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:13:23.170406 21086 heartbeater.cc:347] Connected to a master server at 127.7.94.62:43779
I20260811 00:13:23.170648 21086 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:23.171296 21086 heartbeater.cc:511] Master 127.7.94.62:43779 requested a full tablet report, sending...
I20260811 00:13:23.172609 20458 ts_manager.cc:194] Registered new tserver with Master: 4cab97073ee041138f8cbbab9cb7ac02 (127.7.94.2:43447)
I20260811 00:13:23.173471 20458 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:50385
I20260811 00:13:23.177474 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 20833
I20260811 00:13:23.177613 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:13:23.178014 7544 external_mini_cluster.cc:1428] Setting key 521d96a1b30cc3a65e06d2b52a0269a5
WARNING: no policy specified for oryx/127.7.94.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.94.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.3.keytab.
Entry for principal oryx/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.3.keytab.
I20260811 00:13:23.232113 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:43779
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.3.keytab
--principal=oryx/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:46089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:23.559695 21091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:23.559962 21091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:23.560144 21091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:23.567180 21091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:23.567276 21091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:23.567378 21091 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 00:13:23.567523 21091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:13:23.576920 21091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:46089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.3.keytab
--principal=oryx/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.94.62:43779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:23.578205 21091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:23.579543 21091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:23.592311 21108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:23.592430 21105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:23.592404 21111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:23.593243 21091 server_base.cc:1034] running on GCE node
I20260811 00:13:23.593930 21091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:23.594856 21091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:23.596063 21091 hybrid_clock.cc:648] HybridClock initialized: now 1786407203595990 us; error 53 us; skew 500 ppm
I20260811 00:13:23.596367 21091 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:23 dist-test-slave-8hhm krb5kdc[20383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407203, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:23.599575 21091 init.cc:377] Logged in from keytab as oryx/127.7.94.3@KRBTEST.COM (short username oryx)
I20260811 00:13:23.600914 21091 webserver.cc:460] Webserver started at http://127.7.94.3:40267/ using document root <none> and password file <none>
I20260811 00:13:23.601670 21091 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:23.601822 21091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:23.602142 21091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:23.604923 21091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "0a4ad80f7af04c30bd177b4f3582ed64"
format_stamp: "Formatted at 2026-08-11 00:13:23 on dist-test-slave-8hhm"
server_key: "5eac53a2951cf89490d8234233fa25cf"
server_key_iv: "9c949f66d419cde1e5693be2640d299d"
server_key_version: "encryptionkey@0"
I20260811 00:13:23.605791 21091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "0a4ad80f7af04c30bd177b4f3582ed64"
format_stamp: "Formatted at 2026-08-11 00:13:23 on dist-test-slave-8hhm"
server_key: "5eac53a2951cf89490d8234233fa25cf"
server_key_iv: "9c949f66d419cde1e5693be2640d299d"
server_key_version: "encryptionkey@0"
I20260811 00:13:23.610680 21091 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:13:23.613896 21127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:23.615058 21091 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 00:13:23.615249 21091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "0a4ad80f7af04c30bd177b4f3582ed64"
format_stamp: "Formatted at 2026-08-11 00:13:23 on dist-test-slave-8hhm"
server_key: "5eac53a2951cf89490d8234233fa25cf"
server_key_iv: "9c949f66d419cde1e5693be2640d299d"
server_key_version: "encryptionkey@0"
I20260811 00:13:23.615475 21091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:23.637603 21091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:23.642001 21091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:23.642274 21091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:23.643723 20827 heartbeater.cc:503] Master 127.7.94.62:43779 was elected leader, sending a full tablet report...
I20260811 00:13:23.645437 21091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:23.645618 21091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:23.645828 21091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:23.645915 21091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:13:23 dist-test-slave-8hhm krb5kdc[20383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407203, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM
I20260811 00:13:23.668857 20626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:23.645768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:38825 (local address 127.7.94.62:43779)
0811 00:13:23.645932 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:23.645938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:23.647483 (+ 1545us) server_negotiation.cc:407] Connection header received
0811 00:13:23.649519 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:23.649525 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:23.649631 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:23.649754 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:23.651974 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.652743 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.655134 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.655901 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.656056 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:23.661839 (+ 5783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:23.661847 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:23.661854 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:23.661899 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:23.665753 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:23.666513 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:23.666519 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:23.666522 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:23.666668 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:23.667111 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:23.667133 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:23.667135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:23.667419 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:23.667586 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:23.668265 (+ 679us) server_negotiation.cc:299] Negotiation successful
0811 00:13:23.668494 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 00:13:23.670290 21146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:23.646489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:43779 (local address 127.7.94.3:38825)
0811 00:13:23.647333 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:23.647380 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:23.649203 (+ 1823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:23.650015 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:23.650029 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:23.650870 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:23.651796 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:23.651820 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.652916 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.652928 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:23.654942 (+ 2014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:23.654963 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.656447 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.656458 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:23.656807 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:23.657863 (+ 1056us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:23.657895 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:23.661551 (+ 3656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:23.665910 (+ 4359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:23.665918 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:23.665935 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:23.666385 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:23.666783 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:23.666788 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:23.666792 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:23.666982 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:23.667554 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:23.667564 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:23.667899 (+ 335us) client_negotiation.cc:769] Sending connection context
0811 00:13:23.668171 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 00:13:23.668679 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":333,"threads_started":1}
I20260811 00:13:23.692389 20626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:23.676721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:41071 (local address 127.7.94.62:43779)
0811 00:13:23.676939 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:23.676947 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:23.677949 (+ 1002us) server_negotiation.cc:407] Connection header received
0811 00:13:23.678158 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:23.678161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:23.678229 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:23.678306 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:23.679894 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.680569 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.683099 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.683602 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.683772 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:23.685981 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:23.685990 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:23.685997 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:23.686044 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:23.689423 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:23.690415 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:23.690420 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:23.690424 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:23.690549 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:23.690932 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:23.690939 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:23.690943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:23.691199 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:23.691347 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:23.691781 (+ 434us) server_negotiation.cc:299] Negotiation successful
0811 00:13:23.692003 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:13:23.692452 21249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:23.676711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:43779 (local address 127.7.94.3:41071)
0811 00:13:23.677823 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:23.677848 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:23.678022 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:23.678388 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:23.678395 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:23.679214 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:23.679725 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:23.679742 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.680719 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.680727 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:23.682907 (+ 2180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:23.682931 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.683864 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.683874 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:23.684145 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:23.684951 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:23.684968 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:23.685784 (+ 816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:23.689607 (+ 3823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:23.689612 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:23.689616 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:23.690273 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:23.690671 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:23.690676 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:23.690680 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:23.690820 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:23.691316 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:23.691320 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:23.691495 (+ 175us) client_negotiation.cc:769] Sending connection context
0811 00:13:23.691633 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 00:13:23.691987 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"thread_start_us":720,"threads_started":1}
I20260811 00:13:23.705084 21091 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:45029
I20260811 00:13:23.705102 21339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:45029 every 8 connection(s)
I20260811 00:13:23.706357 21091 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:13:23.708389 21340 heartbeater.cc:347] Connected to a master server at 127.7.94.62:43779
I20260811 00:13:23.708624 21340 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:23.709276 21340 heartbeater.cc:511] Master 127.7.94.62:43779 requested a full tablet report, sending...
I20260811 00:13:23.710561 20458 ts_manager.cc:194] Registered new tserver with Master: 0a4ad80f7af04c30bd177b4f3582ed64 (127.7.94.3:45029)
I20260811 00:13:23.711323 20458 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:38825
I20260811 00:13:23.715425 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 21091
I20260811 00:13:23.715586 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:13:23.716066 7544 external_mini_cluster.cc:1428] Setting key 74867988bf36d2bebaf2096819d00fe5
I20260811 00:13:23.718888 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 00:13:23 dist-test-slave-8hhm krb5kdc[20383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407203, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 00:13:23 dist-test-slave-8hhm krb5kdc[20383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:23 dist-test-slave-8hhm krb5kdc[20383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 00:13:23.775914 20626 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:23.765375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46900 (local address 127.7.94.62:43779)
0811 00:13:23.765592 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:23.765610 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:23.765700 (+ 90us) server_negotiation.cc:407] Connection header received
0811 00:13:23.765935 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:23.765939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:23.766010 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:23.766124 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:23.767519 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.768261 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.769581 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.770023 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.770165 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:23.775673 (+ 5508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46900: BlockingRecv error: recv got EOF from 127.0.0.1:46900 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 00:13:23 dist-test-slave-8hhm krb5kdc[20383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407203, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM
I20260811 00:13:23.791857 20626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:23.779027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46916 (local address 127.7.94.62:43779)
0811 00:13:23.779246 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:23.779252 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:23.779480 (+ 228us) server_negotiation.cc:407] Connection header received
0811 00:13:23.779602 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:23.779605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:23.779694 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:23.779773 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:23.780826 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.781564 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.782665 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.783031 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.783119 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:23.786829 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:23.786834 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:23.786837 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:23.786868 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:23.789309 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:23.789950 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:23.789955 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:23.789959 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:23.790094 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:23.790462 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:23.790467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:23.790480 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:23.790735 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:23.791025 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:23.791308 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 00:13:23.791553 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 00:13:23.796072 20458 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46916:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:13:23.803023 20458 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:13:23.837828 21375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:23.814923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:37117 (local address 127.0.0.1:59750)
0811 00:13:23.815693 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:23.815752 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:23.815971 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:23.827607 (+ 11636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:23.827619 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:23.827663 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:23.828248 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:23.828270 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.830683 (+ 2413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.830692 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:23.834262 (+ 3570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:23.834273 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.836822 (+ 2549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.836828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:23.837031 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:23.837091 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 00:13:23.837253 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 00:13:23.837435 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":1,"thread_start_us":630,"threads_started":1}
I20260811 00:13:23.838666 21378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:23.815384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59750 (local address 127.7.94.1:37117)
0811 00:13:23.826848 (+ 11464us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:23.826860 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:23.826894 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:13:23.827016 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:23.827027 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:23.827335 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:23.827571 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:23.828550 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.830525 (+ 1975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.834416 (+ 3891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.836075 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.836200 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:23.836438 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:23.838235 (+ 1797us) server_negotiation.cc:299] Negotiation successful
0811 00:13:23.838401 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11260,"thread_start_us":12369,"threads_started":1}
I20260811 00:13:23.839679 21377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:23.815105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:45029 (local address 127.0.0.1:46648)
0811 00:13:23.828497 (+ 13392us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:23.828517 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:23.828621 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:23.831714 (+ 3093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:23.831722 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:23.831758 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:23.832700 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:23.832721 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.835408 (+ 2687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.835413 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:23.837380 (+ 1967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:23.837391 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.839054 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.839058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:23.839223 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:23.839246 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 00:13:23.839316 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 00:13:23.839417 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13259,"mutex_wait_us":61,"thread_start_us":13159,"threads_started":1}
I20260811 00:13:23.839920 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:23.815459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46648 (local address 127.7.94.3:45029)
0811 00:13:23.830743 (+ 15284us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:23.830754 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:23.830807 (+ 53us) server_negotiation.cc:407] Connection header received
0811 00:13:23.830924 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:23.830935 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:23.831211 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:23.832262 (+ 1051us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:23.832909 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.835254 (+ 2345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.837525 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.838685 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.838830 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:23.839092 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:23.839404 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 00:13:23.839561 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15070,"thread_start_us":15914,"threads_started":1}
I20260811 00:13:23.842767 20712 tablet_service.cc:1467] Processing CreateTablet for tablet 14d2dc4bfd73477d89ba95111f04083e (DEFAULT_TABLE table=test-table [id=848681aa83624b22af4fa63c9abcbe44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:23.843989 21381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:23.816493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35484 (local address 127.7.94.2:43447)
0811 00:13:23.835737 (+ 19244us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:23.835742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:23.835764 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:13:23.835836 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:23.835840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:23.836027 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:23.836217 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:23.836968 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.838375 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.841608 (+ 3233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.842957 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.843078 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:23.843265 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:23.843536 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 00:13:23.843720 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19085,"thread_start_us":18945,"threads_started":1}
I20260811 00:13:23.843967 21380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:23.815950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:43447 (local address 127.0.0.1:35484)
0811 00:13:23.833111 (+ 17161us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:23.833128 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:23.833259 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:23.836203 (+ 2944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:23.836207 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:23.836231 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:23.836739 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:23.836754 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.838520 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.838525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:23.841410 (+ 2885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:23.841428 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:23.843163 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:23.843169 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:23.843331 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:23.843362 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 00:13:23.843461 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 00:13:23.843636 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17055,"thread_start_us":17388,"threads_started":1}
I20260811 00:13:23.843619 21221 tablet_service.cc:1467] Processing CreateTablet for tablet 14d2dc4bfd73477d89ba95111f04083e (DEFAULT_TABLE table=test-table [id=848681aa83624b22af4fa63c9abcbe44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:23.844345 20712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14d2dc4bfd73477d89ba95111f04083e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:23.844892 21221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14d2dc4bfd73477d89ba95111f04083e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:23.847630 20959 tablet_service.cc:1467] Processing CreateTablet for tablet 14d2dc4bfd73477d89ba95111f04083e (DEFAULT_TABLE table=test-table [id=848681aa83624b22af4fa63c9abcbe44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:23.848860 20959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14d2dc4bfd73477d89ba95111f04083e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:23.855931 21382 tablet_bootstrap.cc:492] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba: Bootstrap starting.
I20260811 00:13:23.860105 21382 tablet_bootstrap.cc:654] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:23.861174 21382 log.cc:824] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba: Log is configured to *not* fsync() on all Append() calls
I20260811 00:13:23.862510 21383 tablet_bootstrap.cc:492] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64: Bootstrap starting.
I20260811 00:13:23.863992 21392 tablet_bootstrap.cc:492] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02: Bootstrap starting.
I20260811 00:13:23.864508 21382 tablet_bootstrap.cc:492] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba: No bootstrap required, opened a new log
I20260811 00:13:23.864820 21382 ts_tablet_manager.cc:1397] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 00:13:23.866056 21383 tablet_bootstrap.cc:654] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:23.866710 21392 tablet_bootstrap.cc:654] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:23.867244 21383 log.cc:824] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64: Log is configured to *not* fsync() on all Append() calls
I20260811 00:13:23.867784 21392 log.cc:824] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02: Log is configured to *not* fsync() on all Append() calls
I20260811 00:13:23.870217 21383 tablet_bootstrap.cc:492] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64: No bootstrap required, opened a new log
I20260811 00:13:23.870494 21383 ts_tablet_manager.cc:1397] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260811 00:13:23.870850 21392 tablet_bootstrap.cc:492] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02: No bootstrap required, opened a new log
I20260811 00:13:23.871100 21392 ts_tablet_manager.cc:1397] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260811 00:13:23.874045 21382 raft_consensus.cc:348] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } }
I20260811 00:13:23.874406 21382 raft_consensus.cc:374] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:23.874532 21382 raft_consensus.cc:729] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5b4b9cf1f114d799efc1e159dcfe7ba, State: Initialized, Role: FOLLOWER
I20260811 00:13:23.875094 21382 consensus_queue.cc:260] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } }
I20260811 00:13:23.876706 21382 ts_tablet_manager.cc:1428] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
I20260811 00:13:23.881201 21383 raft_consensus.cc:348] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } }
I20260811 00:13:23.881309 21392 raft_consensus.cc:348] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } }
I20260811 00:13:23.881671 21392 raft_consensus.cc:374] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:23.881675 21383 raft_consensus.cc:374] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:23.881775 21392 raft_consensus.cc:729] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cab97073ee041138f8cbbab9cb7ac02, State: Initialized, Role: FOLLOWER
I20260811 00:13:23.881775 21383 raft_consensus.cc:729] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a4ad80f7af04c30bd177b4f3582ed64, State: Initialized, Role: FOLLOWER
I20260811 00:13:23.882360 21383 consensus_queue.cc:260] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } }
I20260811 00:13:23.882362 21392 consensus_queue.cc:260] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } }
I20260811 00:13:23.883438 21340 heartbeater.cc:503] Master 127.7.94.62:43779 was elected leader, sending a full tablet report...
I20260811 00:13:23.883438 21086 heartbeater.cc:503] Master 127.7.94.62:43779 was elected leader, sending a full tablet report...
I20260811 00:13:23.884071 21392 ts_tablet_manager.cc:1428] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02: Time spent starting tablet: real 0.013s user 0.011s sys 0.003s
I20260811 00:13:23.884071 21383 ts_tablet_manager.cc:1428] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
W20260811 00:13:23.885792 20828 tablet.cc:2369] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:13:23.920428 21087 tablet.cc:2369] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:13:23.958029 21341 tablet.cc:2369] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:13:24.059522 21400 raft_consensus.cc:482] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:13:24.059888 21400 raft_consensus.cc:504] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } }
I20260811 00:13:24.061697 21400 leader_election.cc:288] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5b4b9cf1f114d799efc1e159dcfe7ba (127.7.94.1:37117), 0a4ad80f7af04c30bd177b4f3582ed64 (127.7.94.3:45029)
I20260811 00:13:24.069301 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:24.062166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:48479 (local address 127.7.94.3:45029)
0811 00:13:24.062429 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:24.062439 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:24.063411 (+ 972us) server_negotiation.cc:407] Connection header received
0811 00:13:24.063698 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:24.063704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:24.063781 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:24.063883 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:24.064760 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.065551 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.067545 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.068295 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.068448 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:24.068563 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:24.068876 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 00:13:24.069005 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 00:13:24.069321 21378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:24.062147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:51321 (local address 127.7.94.1:37117)
0811 00:13:24.062487 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:24.062496 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:24.063194 (+ 698us) server_negotiation.cc:407] Connection header received
0811 00:13:24.063556 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:24.063565 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:24.063654 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:24.063714 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:24.064718 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.065509 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.067493 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.068270 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.068407 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:24.068532 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:24.068924 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 00:13:24.069046 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260811 00:13:24.069449 21404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:24.062134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:45029 (local address 127.7.94.2:48479)
0811 00:13:24.063299 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:24.063315 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:24.063529 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:24.063907 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:24.063913 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:24.063948 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:24.064517 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:24.064542 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.065720 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.065728 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:24.067369 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:24.067383 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.068552 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.068560 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:24.068699 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:24.068741 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 00:13:24.068817 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 00:13:24.068964 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1022,"mutex_wait_us":86,"thread_start_us":1033,"threads_started":1}
I20260811 00:13:24.069512 21403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:24.062010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:37117 (local address 127.7.94.2:51321)
0811 00:13:24.063062 (+ 1052us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:24.063088 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:24.063333 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:24.063907 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:24.063913 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:24.064008 (+ 95us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:24.064557 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:24.064573 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.065663 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.065671 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:24.067336 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:24.067349 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.068520 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.068526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:24.068695 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:24.068766 (+ 71us) client_negotiation.cc:769] Sending connection context
0811 00:13:24.068877 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 00:13:24.069004 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"mutex_wait_us":137,"spinlock_wait_cycles":24960,"thread_start_us":696,"threads_started":1}
I20260811 00:13:24.070405 20753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d2dc4bfd73477d89ba95111f04083e" candidate_uuid: "4cab97073ee041138f8cbbab9cb7ac02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" is_pre_election: true
I20260811 00:13:24.070408 21264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d2dc4bfd73477d89ba95111f04083e" candidate_uuid: "4cab97073ee041138f8cbbab9cb7ac02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" is_pre_election: true
I20260811 00:13:24.070909 21264 raft_consensus.cc:2454] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cab97073ee041138f8cbbab9cb7ac02 in term 0.
I20260811 00:13:24.070922 20753 raft_consensus.cc:2454] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cab97073ee041138f8cbbab9cb7ac02 in term 0.
I20260811 00:13:24.071592 20869 leader_election.cc:302] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0a4ad80f7af04c30bd177b4f3582ed64, 4cab97073ee041138f8cbbab9cb7ac02, f5b4b9cf1f114d799efc1e159dcfe7ba; no voters:
I20260811 00:13:24.072016 21400 raft_consensus.cc:2781] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:13:24.072218 21400 raft_consensus.cc:482] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:13:24.072302 21400 raft_consensus.cc:3037] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:24.075240 21400 raft_consensus.cc:504] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } }
I20260811 00:13:24.075809 21400 leader_election.cc:288] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [CANDIDATE]: Term 1 election: Requested vote from peers f5b4b9cf1f114d799efc1e159dcfe7ba (127.7.94.1:37117), 0a4ad80f7af04c30bd177b4f3582ed64 (127.7.94.3:45029)
I20260811 00:13:24.076249 20753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d2dc4bfd73477d89ba95111f04083e" candidate_uuid: "4cab97073ee041138f8cbbab9cb7ac02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba"
I20260811 00:13:24.076267 21264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14d2dc4bfd73477d89ba95111f04083e" candidate_uuid: "4cab97073ee041138f8cbbab9cb7ac02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a4ad80f7af04c30bd177b4f3582ed64"
I20260811 00:13:24.076488 20753 raft_consensus.cc:3037] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:24.076499 21264 raft_consensus.cc:3037] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:24.078842 21264 raft_consensus.cc:2454] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cab97073ee041138f8cbbab9cb7ac02 in term 1.
I20260811 00:13:24.078861 20753 raft_consensus.cc:2454] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cab97073ee041138f8cbbab9cb7ac02 in term 1.
I20260811 00:13:24.079244 20873 leader_election.cc:302] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a4ad80f7af04c30bd177b4f3582ed64, 4cab97073ee041138f8cbbab9cb7ac02; no voters:
I20260811 00:13:24.079555 21400 raft_consensus.cc:2781] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:13:24.079998 21400 raft_consensus.cc:686] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [term 1 LEADER]: Becoming Leader. State: Replica: 4cab97073ee041138f8cbbab9cb7ac02, State: Running, Role: LEADER
I20260811 00:13:24.080554 21400 consensus_queue.cc:237] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } }
I20260811 00:13:24.085426 20453 catalog_manager.cc:5495] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cab97073ee041138f8cbbab9cb7ac02 (127.7.94.2). New cstate: current_term: 1 leader_uuid: "4cab97073ee041138f8cbbab9cb7ac02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } health_report { overall_health: HEALTHY } } }
I20260811 00:13:24.132871 21381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:24.127569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35488 (local address 127.7.94.2:43447)
0811 00:13:24.127794 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:24.127799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:24.127820 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:13:24.127966 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:24.127970 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:24.128095 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:24.128251 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:13:24.128606 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.129518 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.130831 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.131244 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.131379 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:24.131798 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:13:24.132305 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:13:24.132471 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:24.132523 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 00:13:24.132601 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 00:13:24.141033 21264 raft_consensus.cc:1261] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64 [term 1 FOLLOWER]: Refusing update from remote peer 4cab97073ee041138f8cbbab9cb7ac02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:13:24.141036 20753 raft_consensus.cc:1261] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba [term 1 FOLLOWER]: Refusing update from remote peer 4cab97073ee041138f8cbbab9cb7ac02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:13:24.142114 21409 consensus_queue.cc:1035] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:13:24.142450 21400 consensus_queue.cc:1035] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:13:24.157889 21413 mvcc.cc:205] Tried to move back new op lower bound from 7317123908151988224 to 7317123907918446592. Current Snapshot: MvccSnapshot[applied={T|T < 7317123908151988224}]
I20260811 00:13:24.161266 21414 mvcc.cc:205] Tried to move back new op lower bound from 7317123908151988224 to 7317123907918446592. Current Snapshot: MvccSnapshot[applied={T|T < 7317123908151988224}]
I20260811 00:13:24.181855 20453 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:13:24.182153 20453 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:13:24.185940 20453 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f: Sending DeleteTablet for 3 replicas of tablet 14d2dc4bfd73477d89ba95111f04083e
I20260811 00:13:24.187077 20712 tablet_service.cc:1514] Processing DeleteTablet for tablet 14d2dc4bfd73477d89ba95111f04083e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:24 UTC) from {username='oryx', principal='oryx/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:59750
I20260811 00:13:24.187161 20959 tablet_service.cc:1514] Processing DeleteTablet for tablet 14d2dc4bfd73477d89ba95111f04083e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:24 UTC) from {username='oryx', principal='oryx/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:35484
I20260811 00:13:24.187433 21221 tablet_service.cc:1514] Processing DeleteTablet for tablet 14d2dc4bfd73477d89ba95111f04083e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:24 UTC) from {username='oryx', principal='oryx/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:46648
I20260811 00:13:24.189512 20453 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46916:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:13:24.190366 20453 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:13:24.199757 20711 tablet_service.cc:1467] Processing CreateTablet for tablet a4cfabd8f504414bbe01f5a356afd192 (DEFAULT_TABLE table=test-table [id=11d770a1ef364a1e8be6dfd618854687]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:24.199753 20957 tablet_service.cc:1467] Processing CreateTablet for tablet a4cfabd8f504414bbe01f5a356afd192 (DEFAULT_TABLE table=test-table [id=11d770a1ef364a1e8be6dfd618854687]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:24.200361 20711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4cfabd8f504414bbe01f5a356afd192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:24.200501 20957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4cfabd8f504414bbe01f5a356afd192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:24.208142 21218 tablet_service.cc:1467] Processing CreateTablet for tablet a4cfabd8f504414bbe01f5a356afd192 (DEFAULT_TABLE table=test-table [id=11d770a1ef364a1e8be6dfd618854687]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:24.209877 21218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4cfabd8f504414bbe01f5a356afd192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:24.214680 21382 tablet_bootstrap.cc:492] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba: Bootstrap starting.
I20260811 00:13:24.216543 21382 tablet_bootstrap.cc:654] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:24.228641 21382 tablet_bootstrap.cc:492] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba: No bootstrap required, opened a new log
I20260811 00:13:24.228835 21382 ts_tablet_manager.cc:1397] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260811 00:13:24.230286 21382 raft_consensus.cc:348] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:24.230549 21382 raft_consensus.cc:374] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:24.230674 21382 raft_consensus.cc:729] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5b4b9cf1f114d799efc1e159dcfe7ba, State: Initialized, Role: FOLLOWER
I20260811 00:13:24.231029 21382 consensus_queue.cc:260] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:24.231745 21382 ts_tablet_manager.cc:1428] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 00:13:24.240756 21383 tablet_bootstrap.cc:492] T a4cfabd8f504414bbe01f5a356afd192 P 0a4ad80f7af04c30bd177b4f3582ed64: Bootstrap starting.
I20260811 00:13:24.243254 21383 tablet_bootstrap.cc:654] T a4cfabd8f504414bbe01f5a356afd192 P 0a4ad80f7af04c30bd177b4f3582ed64: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:24.249580 21392 tablet_bootstrap.cc:492] T a4cfabd8f504414bbe01f5a356afd192 P 4cab97073ee041138f8cbbab9cb7ac02: Bootstrap starting.
I20260811 00:13:24.250204 21383 tablet_bootstrap.cc:492] T a4cfabd8f504414bbe01f5a356afd192 P 0a4ad80f7af04c30bd177b4f3582ed64: No bootstrap required, opened a new log
I20260811 00:13:24.250391 21383 ts_tablet_manager.cc:1397] T a4cfabd8f504414bbe01f5a356afd192 P 0a4ad80f7af04c30bd177b4f3582ed64: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260811 00:13:24.251546 21383 raft_consensus.cc:348] T a4cfabd8f504414bbe01f5a356afd192 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:24.251919 21383 raft_consensus.cc:374] T a4cfabd8f504414bbe01f5a356afd192 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:24.252043 21383 raft_consensus.cc:729] T a4cfabd8f504414bbe01f5a356afd192 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a4ad80f7af04c30bd177b4f3582ed64, State: Initialized, Role: FOLLOWER
I20260811 00:13:24.252487 21392 tablet_bootstrap.cc:654] T a4cfabd8f504414bbe01f5a356afd192 P 4cab97073ee041138f8cbbab9cb7ac02: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:24.252380 21383 consensus_queue.cc:260] T a4cfabd8f504414bbe01f5a356afd192 P 0a4ad80f7af04c30bd177b4f3582ed64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:24.253315 21383 ts_tablet_manager.cc:1428] T a4cfabd8f504414bbe01f5a356afd192 P 0a4ad80f7af04c30bd177b4f3582ed64: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260811 00:13:24.277822 21392 tablet_bootstrap.cc:492] T a4cfabd8f504414bbe01f5a356afd192 P 4cab97073ee041138f8cbbab9cb7ac02: No bootstrap required, opened a new log
I20260811 00:13:24.278062 21392 ts_tablet_manager.cc:1397] T a4cfabd8f504414bbe01f5a356afd192 P 4cab97073ee041138f8cbbab9cb7ac02: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.007s
I20260811 00:13:24.279261 21392 raft_consensus.cc:348] T a4cfabd8f504414bbe01f5a356afd192 P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:24.279593 21392 raft_consensus.cc:374] T a4cfabd8f504414bbe01f5a356afd192 P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:24.279733 21392 raft_consensus.cc:729] T a4cfabd8f504414bbe01f5a356afd192 P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cab97073ee041138f8cbbab9cb7ac02, State: Initialized, Role: FOLLOWER
I20260811 00:13:24.280138 21392 consensus_queue.cc:260] T a4cfabd8f504414bbe01f5a356afd192 P 4cab97073ee041138f8cbbab9cb7ac02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:24.285739 21392 ts_tablet_manager.cc:1428] T a4cfabd8f504414bbe01f5a356afd192 P 4cab97073ee041138f8cbbab9cb7ac02: Time spent starting tablet: real 0.008s user 0.000s sys 0.003s
I20260811 00:13:24.289043 21436 tablet_replica.cc:330] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02: stopping tablet replica
I20260811 00:13:24.289651 21436 raft_consensus.cc:2229] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:13:24.306118 21436 raft_consensus.cc:2258] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:24.307168 21435 tablet_replica.cc:330] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba: stopping tablet replica
I20260811 00:13:24.307781 21435 raft_consensus.cc:2229] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:13:24.308372 21435 raft_consensus.cc:2258] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:24.308686 21436 ts_tablet_manager.cc:1905] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:24.309242 21437 tablet_replica.cc:330] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64: stopping tablet replica
I20260811 00:13:24.309753 21437 raft_consensus.cc:2229] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:13:24.310185 21437 raft_consensus.cc:2258] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:24.314990 21437 ts_tablet_manager.cc:1905] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:24.319320 21435 ts_tablet_manager.cc:1905] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:24.319613 21436 ts_tablet_manager.cc:1918] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:13:24.319836 21436 log.cc:1196] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/14d2dc4bfd73477d89ba95111f04083e
I20260811 00:13:24.320508 21436 ts_tablet_manager.cc:1939] T 14d2dc4bfd73477d89ba95111f04083e P 4cab97073ee041138f8cbbab9cb7ac02: Deleting consensus metadata
I20260811 00:13:24.322824 21437 ts_tablet_manager.cc:1918] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:13:24.323012 21437 log.cc:1196] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/14d2dc4bfd73477d89ba95111f04083e
I20260811 00:13:24.323529 21437 ts_tablet_manager.cc:1939] T 14d2dc4bfd73477d89ba95111f04083e P 0a4ad80f7af04c30bd177b4f3582ed64: Deleting consensus metadata
I20260811 00:13:24.324909 20435 catalog_manager.cc:4841] TS 4cab97073ee041138f8cbbab9cb7ac02 (127.7.94.2:43447): tablet 14d2dc4bfd73477d89ba95111f04083e (table test-table [id=848681aa83624b22af4fa63c9abcbe44]) successfully deleted
I20260811 00:13:24.326301 21435 ts_tablet_manager.cc:1918] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:13:24.326462 21435 log.cc:1196] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/14d2dc4bfd73477d89ba95111f04083e
I20260811 00:13:24.327029 21435 ts_tablet_manager.cc:1939] T 14d2dc4bfd73477d89ba95111f04083e P f5b4b9cf1f114d799efc1e159dcfe7ba: Deleting consensus metadata
I20260811 00:13:24.327014 20434 catalog_manager.cc:4841] TS 0a4ad80f7af04c30bd177b4f3582ed64 (127.7.94.3:45029): tablet 14d2dc4bfd73477d89ba95111f04083e (table test-table [id=848681aa83624b22af4fa63c9abcbe44]) successfully deleted
I20260811 00:13:24.329051 20433 catalog_manager.cc:4841] TS f5b4b9cf1f114d799efc1e159dcfe7ba (127.7.94.1:37117): tablet 14d2dc4bfd73477d89ba95111f04083e (table test-table [id=848681aa83624b22af4fa63c9abcbe44]) successfully deleted
I20260811 00:13:24.532740 21399 raft_consensus.cc:482] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:13:24.532968 21399 raft_consensus.cc:504] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:24.534766 21399 leader_election.cc:288] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cab97073ee041138f8cbbab9cb7ac02 (127.7.94.2:43447), 0a4ad80f7af04c30bd177b4f3582ed64 (127.7.94.3:45029)
I20260811 00:13:24.542742 21381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:24.535200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:38241 (local address 127.7.94.2:43447)
0811 00:13:24.535555 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:24.535561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:24.535895 (+ 334us) server_negotiation.cc:407] Connection header received
0811 00:13:24.536238 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:24.536251 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:24.536343 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:24.536404 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:24.537259 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.538423 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.540526 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.541595 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.541735 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:24.541834 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:24.542208 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 00:13:24.542360 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186}
I20260811 00:13:24.543018 21445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:24.534976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:43447 (local address 127.7.94.1:38241)
0811 00:13:24.535767 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:24.535792 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:24.536044 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:24.536634 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:24.536640 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:24.536669 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:24.537016 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:24.537029 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.538589 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.538594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:24.540355 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:24.540368 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.541837 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.541844 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:24.542020 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:24.542058 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 00:13:24.542166 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 00:13:24.542301 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":1,"thread_start_us":1327,"threads_started":1}
I20260811 00:13:24.543542 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:24.535734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:36767 (local address 127.7.94.3:45029)
0811 00:13:24.535927 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:24.535933 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:24.536796 (+ 863us) server_negotiation.cc:407] Connection header received
0811 00:13:24.536999 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:24.537004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:24.537111 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:24.537217 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:24.537769 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.538478 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.541104 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.542183 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.542300 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:24.542405 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:24.543147 (+ 742us) server_negotiation.cc:299] Negotiation successful
0811 00:13:24.543277 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:13:24.543684 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:24.535130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:45029 (local address 127.7.94.1:36767)
0811 00:13:24.536657 (+ 1527us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:24.536680 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:24.536834 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:24.537282 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:24.537286 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:24.537312 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:24.537614 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:24.537622 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.538639 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.538645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:24.540931 (+ 2286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:24.540948 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.542574 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.542580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:24.542872 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:24.542909 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 00:13:24.543111 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 00:13:24.543238 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1390,"spinlock_wait_cycles":1792,"thread_start_us":879,"threads_started":1}
I20260811 00:13:24.544596 21264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4cfabd8f504414bbe01f5a356afd192" candidate_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" is_pre_election: true
I20260811 00:13:24.544668 21004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4cfabd8f504414bbe01f5a356afd192" candidate_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cab97073ee041138f8cbbab9cb7ac02" is_pre_election: true
I20260811 00:13:24.544878 21264 raft_consensus.cc:2454] T a4cfabd8f504414bbe01f5a356afd192 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5b4b9cf1f114d799efc1e159dcfe7ba in term 0.
I20260811 00:13:24.545235 21004 raft_consensus.cc:2454] T a4cfabd8f504414bbe01f5a356afd192 P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5b4b9cf1f114d799efc1e159dcfe7ba in term 0.
I20260811 00:13:24.545450 20618 leader_election.cc:302] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a4ad80f7af04c30bd177b4f3582ed64, f5b4b9cf1f114d799efc1e159dcfe7ba; no voters:
I20260811 00:13:24.546132 21399 raft_consensus.cc:2781] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:13:24.546267 21399 raft_consensus.cc:482] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:13:24.546370 21399 raft_consensus.cc:3037] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:24.549706 21399 raft_consensus.cc:504] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:24.550437 21399 leader_election.cc:288] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [CANDIDATE]: Term 1 election: Requested vote from peers 4cab97073ee041138f8cbbab9cb7ac02 (127.7.94.2:43447), 0a4ad80f7af04c30bd177b4f3582ed64 (127.7.94.3:45029)
I20260811 00:13:24.550920 21264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4cfabd8f504414bbe01f5a356afd192" candidate_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a4ad80f7af04c30bd177b4f3582ed64"
I20260811 00:13:24.550915 21004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4cfabd8f504414bbe01f5a356afd192" candidate_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cab97073ee041138f8cbbab9cb7ac02"
I20260811 00:13:24.551126 21004 raft_consensus.cc:3037] T a4cfabd8f504414bbe01f5a356afd192 P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:24.551139 21264 raft_consensus.cc:3037] T a4cfabd8f504414bbe01f5a356afd192 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:24.553175 21264 raft_consensus.cc:2454] T a4cfabd8f504414bbe01f5a356afd192 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5b4b9cf1f114d799efc1e159dcfe7ba in term 1.
I20260811 00:13:24.553654 20618 leader_election.cc:302] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a4ad80f7af04c30bd177b4f3582ed64, f5b4b9cf1f114d799efc1e159dcfe7ba; no voters:
I20260811 00:13:24.553951 21399 raft_consensus.cc:2781] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:13:24.554469 21399 raft_consensus.cc:686] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 1 LEADER]: Becoming Leader. State: Replica: f5b4b9cf1f114d799efc1e159dcfe7ba, State: Running, Role: LEADER
I20260811 00:13:24.554598 21004 raft_consensus.cc:2454] T a4cfabd8f504414bbe01f5a356afd192 P 4cab97073ee041138f8cbbab9cb7ac02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5b4b9cf1f114d799efc1e159dcfe7ba in term 1.
I20260811 00:13:24.554903 21399 consensus_queue.cc:237] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:24.558547 20453 catalog_manager.cc:5495] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba reported cstate change: term changed from 0 to 1, leader changed from <none> to f5b4b9cf1f114d799efc1e159dcfe7ba (127.7.94.1). New cstate: current_term: 1 leader_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } health_report { overall_health: UNKNOWN } } }
Aug 11 00:13:24 dist-test-slave-8hhm krb5kdc[20383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407201, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.62@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM
I20260811 00:13:24.632395 21457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:24.615712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46926 (local address 127.7.94.62:43779)
0811 00:13:24.616749 (+ 1037us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:24.616754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:24.616774 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:13:24.616825 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:24.616828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:24.616880 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:24.616966 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:24.618491 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.619328 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.621261 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.621664 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.621755 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:24.625752 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:24.625757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:24.625760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:24.625787 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:24.628683 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:24.629655 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:24.629659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:24.629662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:24.629825 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:24.630250 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:24.630254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:24.630256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:24.630499 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:24.630626 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:24.631450 (+ 824us) server_negotiation.cc:299] Negotiation successful
0811 00:13:24.631801 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":135,"server-negotiator.queue_time_us":941,"thread_start_us":791,"threads_started":1}
I20260811 00:13:24.633474 21458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:24.615652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:43779 (local address 127.0.0.1:46926)
0811 00:13:24.616443 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:24.616468 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:24.616648 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:24.617052 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:24.617057 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:24.618069 (+ 1012us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:24.618342 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:24.618356 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.619497 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.619503 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:24.621085 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:24.621101 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.621842 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.621850 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:24.622033 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:24.622720 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:24.622743 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:24.625574 (+ 2831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:24.628815 (+ 3241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:24.628829 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:24.628838 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:24.629532 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:24.629950 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:24.629956 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:24.629960 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:24.630134 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:24.631049 (+ 915us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:24.631054 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:24.631206 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 00:13:24.631364 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 00:13:24.631646 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":385,"threads_started":1}
I20260811 00:13:24.638718 20453 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:46926:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 00:13:24.639891 20453 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:13:24.646993 20712 tablet_service.cc:1467] Processing CreateTablet for tablet 288e3a7f002643a99c56bfa4228f5b33 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8bb5e56d9f8b44f0b2e522dfc1c31998]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:13:24.647562 20712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 288e3a7f002643a99c56bfa4228f5b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:24.647559 21221 tablet_service.cc:1467] Processing CreateTablet for tablet 288e3a7f002643a99c56bfa4228f5b33 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8bb5e56d9f8b44f0b2e522dfc1c31998]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:13:24.648161 21221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 288e3a7f002643a99c56bfa4228f5b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:24.653782 21383 tablet_bootstrap.cc:492] T 288e3a7f002643a99c56bfa4228f5b33 P 0a4ad80f7af04c30bd177b4f3582ed64: Bootstrap starting.
I20260811 00:13:24.653931 20959 tablet_service.cc:1467] Processing CreateTablet for tablet 288e3a7f002643a99c56bfa4228f5b33 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8bb5e56d9f8b44f0b2e522dfc1c31998]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 00:13:24.654503 20959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 288e3a7f002643a99c56bfa4228f5b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:24.655557 21383 tablet_bootstrap.cc:654] T 288e3a7f002643a99c56bfa4228f5b33 P 0a4ad80f7af04c30bd177b4f3582ed64: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:24.660547 21382 tablet_bootstrap.cc:492] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba: Bootstrap starting.
I20260811 00:13:24.662330 21382 tablet_bootstrap.cc:654] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:24.663038 21392 tablet_bootstrap.cc:492] T 288e3a7f002643a99c56bfa4228f5b33 P 4cab97073ee041138f8cbbab9cb7ac02: Bootstrap starting.
I20260811 00:13:24.665041 21392 tablet_bootstrap.cc:654] T 288e3a7f002643a99c56bfa4228f5b33 P 4cab97073ee041138f8cbbab9cb7ac02: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:24.672482 21382 tablet_bootstrap.cc:492] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba: No bootstrap required, opened a new log
I20260811 00:13:24.672657 21382 ts_tablet_manager.cc:1397] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.004s
I20260811 00:13:24.673441 21382 raft_consensus.cc:348] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:24.673702 21382 raft_consensus.cc:374] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:24.673781 21382 raft_consensus.cc:729] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5b4b9cf1f114d799efc1e159dcfe7ba, State: Initialized, Role: FOLLOWER
I20260811 00:13:24.674101 21382 consensus_queue.cc:260] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:24.674623 21399 tablet_replica.cc:439] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } } }
I20260811 00:13:24.674909 21399 tablet_replica.cc:442] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:13:24.675392 21382 ts_tablet_manager.cc:1428] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 00:13:24.677294 21383 tablet_bootstrap.cc:492] T 288e3a7f002643a99c56bfa4228f5b33 P 0a4ad80f7af04c30bd177b4f3582ed64: No bootstrap required, opened a new log
I20260811 00:13:24.677482 21383 ts_tablet_manager.cc:1397] T 288e3a7f002643a99c56bfa4228f5b33 P 0a4ad80f7af04c30bd177b4f3582ed64: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.014s
I20260811 00:13:24.678431 21383 raft_consensus.cc:348] T 288e3a7f002643a99c56bfa4228f5b33 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:24.678682 21383 raft_consensus.cc:374] T 288e3a7f002643a99c56bfa4228f5b33 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:24.678782 21383 raft_consensus.cc:729] T 288e3a7f002643a99c56bfa4228f5b33 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a4ad80f7af04c30bd177b4f3582ed64, State: Initialized, Role: FOLLOWER
I20260811 00:13:24.679057 21383 consensus_queue.cc:260] T 288e3a7f002643a99c56bfa4228f5b33 P 0a4ad80f7af04c30bd177b4f3582ed64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:24.679481 21401 tablet_replica.cc:439] T 288e3a7f002643a99c56bfa4228f5b33 P 0a4ad80f7af04c30bd177b4f3582ed64: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } } }
I20260811 00:13:24.679853 21401 tablet_replica.cc:442] T 288e3a7f002643a99c56bfa4228f5b33 P 0a4ad80f7af04c30bd177b4f3582ed64: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:13:24.684298 21383 ts_tablet_manager.cc:1428] T 288e3a7f002643a99c56bfa4228f5b33 P 0a4ad80f7af04c30bd177b4f3582ed64: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260811 00:13:24.685468 21392 tablet_bootstrap.cc:492] T 288e3a7f002643a99c56bfa4228f5b33 P 4cab97073ee041138f8cbbab9cb7ac02: No bootstrap required, opened a new log
I20260811 00:13:24.685667 21392 ts_tablet_manager.cc:1397] T 288e3a7f002643a99c56bfa4228f5b33 P 4cab97073ee041138f8cbbab9cb7ac02: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.014s
I20260811 00:13:24.686580 21392 raft_consensus.cc:348] T 288e3a7f002643a99c56bfa4228f5b33 P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:24.686880 21392 raft_consensus.cc:374] T 288e3a7f002643a99c56bfa4228f5b33 P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:24.686983 21392 raft_consensus.cc:729] T 288e3a7f002643a99c56bfa4228f5b33 P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cab97073ee041138f8cbbab9cb7ac02, State: Initialized, Role: FOLLOWER
I20260811 00:13:24.687247 21392 consensus_queue.cc:260] T 288e3a7f002643a99c56bfa4228f5b33 P 4cab97073ee041138f8cbbab9cb7ac02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:24.687845 21392 ts_tablet_manager.cc:1428] T 288e3a7f002643a99c56bfa4228f5b33 P 4cab97073ee041138f8cbbab9cb7ac02: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 00:13:24.687765 21420 tablet_replica.cc:439] T 288e3a7f002643a99c56bfa4228f5b33 P 4cab97073ee041138f8cbbab9cb7ac02: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } } }
I20260811 00:13:24.688055 21420 tablet_replica.cc:442] T 288e3a7f002643a99c56bfa4228f5b33 P 4cab97073ee041138f8cbbab9cb7ac02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:13:24.779475 21399 raft_consensus.cc:482] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:13:24.779763 21399 raft_consensus.cc:504] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:24.780416 21399 leader_election.cc:288] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cab97073ee041138f8cbbab9cb7ac02 (127.7.94.2:43447), 0a4ad80f7af04c30bd177b4f3582ed64 (127.7.94.3:45029)
I20260811 00:13:24.780936 21004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "288e3a7f002643a99c56bfa4228f5b33" candidate_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cab97073ee041138f8cbbab9cb7ac02" is_pre_election: true
I20260811 00:13:24.781021 21264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "288e3a7f002643a99c56bfa4228f5b33" candidate_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" is_pre_election: true
I20260811 00:13:24.781252 21004 raft_consensus.cc:2454] T 288e3a7f002643a99c56bfa4228f5b33 P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5b4b9cf1f114d799efc1e159dcfe7ba in term 0.
I20260811 00:13:24.781517 21264 raft_consensus.cc:2454] T 288e3a7f002643a99c56bfa4228f5b33 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5b4b9cf1f114d799efc1e159dcfe7ba in term 0.
I20260811 00:13:24.781873 20619 leader_election.cc:302] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cab97073ee041138f8cbbab9cb7ac02, f5b4b9cf1f114d799efc1e159dcfe7ba; no voters:
I20260811 00:13:24.782220 21399 raft_consensus.cc:2781] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:13:24.782316 21399 raft_consensus.cc:482] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:13:24.782399 21399 raft_consensus.cc:3037] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:24.784365 21399 raft_consensus.cc:504] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:24.784922 21399 leader_election.cc:288] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba [CANDIDATE]: Term 1 election: Requested vote from peers 4cab97073ee041138f8cbbab9cb7ac02 (127.7.94.2:43447), 0a4ad80f7af04c30bd177b4f3582ed64 (127.7.94.3:45029)
I20260811 00:13:24.785347 21004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "288e3a7f002643a99c56bfa4228f5b33" candidate_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cab97073ee041138f8cbbab9cb7ac02"
I20260811 00:13:24.785383 21264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "288e3a7f002643a99c56bfa4228f5b33" candidate_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a4ad80f7af04c30bd177b4f3582ed64"
I20260811 00:13:24.785583 21004 raft_consensus.cc:3037] T 288e3a7f002643a99c56bfa4228f5b33 P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:24.785635 21264 raft_consensus.cc:3037] T 288e3a7f002643a99c56bfa4228f5b33 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:24.787588 21264 raft_consensus.cc:2454] T 288e3a7f002643a99c56bfa4228f5b33 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5b4b9cf1f114d799efc1e159dcfe7ba in term 1.
I20260811 00:13:24.787756 21004 raft_consensus.cc:2454] T 288e3a7f002643a99c56bfa4228f5b33 P 4cab97073ee041138f8cbbab9cb7ac02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5b4b9cf1f114d799efc1e159dcfe7ba in term 1.
I20260811 00:13:24.788185 20618 leader_election.cc:302] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a4ad80f7af04c30bd177b4f3582ed64, f5b4b9cf1f114d799efc1e159dcfe7ba; no voters:
I20260811 00:13:24.788496 21399 raft_consensus.cc:2781] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:13:24.788709 21399 raft_consensus.cc:686] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 1 LEADER]: Becoming Leader. State: Replica: f5b4b9cf1f114d799efc1e159dcfe7ba, State: Running, Role: LEADER
I20260811 00:13:24.788970 21399 consensus_queue.cc:237] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:24.790714 21446 tablet_replica.cc:439] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba: TxnStatusTablet state changed. Reason: New leader f5b4b9cf1f114d799efc1e159dcfe7ba. Latest consensus state: current_term: 1 leader_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } } }
I20260811 00:13:24.791041 21446 tablet_replica.cc:442] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:13:24.791764 21464 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 00:13:24.792033 21464 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 00:13:24.792433 20458 catalog_manager.cc:5495] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba reported cstate change: term changed from 0 to 1, leader changed from <none> to f5b4b9cf1f114d799efc1e159dcfe7ba (127.7.94.1). New cstate: current_term: 1 leader_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } health_report { overall_health: UNKNOWN } } }
I20260811 00:13:24.817860 21458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:24.811534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:37117 (local address 127.0.0.1:59766)
0811 00:13:24.811790 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:24.811813 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:24.811971 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:24.812646 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:24.812652 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:24.812669 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:13:24.812888 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:24.812903 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.814126 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.814132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:24.815128 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:24.815139 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.815922 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.815928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:24.816089 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:24.816146 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:13:24.817250 (+ 1104us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:13:24.817273 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 00:13:24.817369 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 00:13:24.817522 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 00:13:24.817898 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:24.811560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59766 (local address 127.7.94.1:37117)
0811 00:13:24.812246 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:24.812251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:24.812269 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:13:24.812345 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:24.812349 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:24.812434 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:24.812602 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:13:24.813074 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.813992 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.815281 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.815755 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.815888 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:24.816370 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:13:24.817111 (+ 741us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:13:24.817303 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:24.817409 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 00:13:24.817492 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":370,"threads_started":1}
I20260811 00:13:24.822685 21264 raft_consensus.cc:1261] T 288e3a7f002643a99c56bfa4228f5b33 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 1 FOLLOWER]: Refusing update from remote peer f5b4b9cf1f114d799efc1e159dcfe7ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:13:24.823235 21004 raft_consensus.cc:1261] T 288e3a7f002643a99c56bfa4228f5b33 P 4cab97073ee041138f8cbbab9cb7ac02 [term 1 FOLLOWER]: Refusing update from remote peer f5b4b9cf1f114d799efc1e159dcfe7ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:13:24.823432 21399 consensus_queue.cc:1035] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:13:24.824072 21399 consensus_queue.cc:1035] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:13:24.828586 21420 tablet_replica.cc:439] T 288e3a7f002643a99c56bfa4228f5b33 P 4cab97073ee041138f8cbbab9cb7ac02: TxnStatusTablet state changed. Reason: New leader f5b4b9cf1f114d799efc1e159dcfe7ba. Latest consensus state: current_term: 1 leader_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } } }
I20260811 00:13:24.828881 21420 tablet_replica.cc:442] T 288e3a7f002643a99c56bfa4228f5b33 P 4cab97073ee041138f8cbbab9cb7ac02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:13:24.834985 21469 mvcc.cc:205] Tried to move back new op lower bound from 7317123910946230272 to 7317123910818516992. Current Snapshot: MvccSnapshot[applied={T|T < 7317123910946230272}]
I20260811 00:13:24.839293 21420 tablet_replica.cc:439] T 288e3a7f002643a99c56bfa4228f5b33 P 4cab97073ee041138f8cbbab9cb7ac02: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } } }
I20260811 00:13:24.839650 21420 tablet_replica.cc:442] T 288e3a7f002643a99c56bfa4228f5b33 P 4cab97073ee041138f8cbbab9cb7ac02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:13:24.840134 21401 tablet_replica.cc:439] T 288e3a7f002643a99c56bfa4228f5b33 P 0a4ad80f7af04c30bd177b4f3582ed64: TxnStatusTablet state changed. Reason: New leader f5b4b9cf1f114d799efc1e159dcfe7ba. Latest consensus state: current_term: 1 leader_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } } }
I20260811 00:13:24.840435 21401 tablet_replica.cc:442] T 288e3a7f002643a99c56bfa4228f5b33 P 0a4ad80f7af04c30bd177b4f3582ed64: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:13:24.844899 21399 tablet_replica.cc:439] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } } }
I20260811 00:13:24.845359 21399 tablet_replica.cc:442] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:13:24.845683 21401 tablet_replica.cc:439] T 288e3a7f002643a99c56bfa4228f5b33 P 0a4ad80f7af04c30bd177b4f3582ed64: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } } }
I20260811 00:13:24.845927 21401 tablet_replica.cc:442] T 288e3a7f002643a99c56bfa4228f5b33 P 0a4ad80f7af04c30bd177b4f3582ed64: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 00:13:24.848613 21399 tablet_replica.cc:439] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } } }
I20260811 00:13:24.848870 21399 tablet_replica.cc:442] T 288e3a7f002643a99c56bfa4228f5b33 P f5b4b9cf1f114d799efc1e159dcfe7ba: This TxnStatusTablet replica's current role is: LEADER
I20260811 00:13:24.856680 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:24.851089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59770 (local address 127.7.94.1:37117)
0811 00:13:24.851313 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:24.851321 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:24.851877 (+ 556us) server_negotiation.cc:407] Connection header received
0811 00:13:24.852077 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:24.852082 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:24.852158 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:24.852266 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:13:24.852735 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.853560 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.854909 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.855381 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.855518 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:24.855926 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:13:24.856024 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:13:24.856133 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:24.856294 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 00:13:24.856371 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 00:13:24.873718 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:24.867705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:44333 (local address 127.7.94.1:37117)
0811 00:13:24.867862 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:24.867866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:24.868555 (+ 689us) server_negotiation.cc:407] Connection header received
0811 00:13:24.868789 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:24.868793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:24.868855 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:24.868949 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:13:24.869540 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.870281 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.871726 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.872249 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.872397 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:24.872843 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:13:24.872981 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:13:24.873081 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:24.873255 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 00:13:24.873326 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":60}
I20260811 00:13:24.873790 21489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:24.867817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:37117 (local address 127.7.94.1:44333)
0811 00:13:24.868458 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:24.868474 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:24.868634 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:24.868964 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:24.868968 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:24.868978 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:13:24.869360 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:24.869371 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.870425 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.870430 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:24.871567 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:24.871582 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:24.872473 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:24.872482 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:24.872610 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:24.872663 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:13:24.873081 (+ 418us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:13:24.873094 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 00:13:24.873240 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 00:13:24.873364 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":378,"threads_started":1}
I20260811 00:13:24.884919 21264 raft_consensus.cc:1261] T a4cfabd8f504414bbe01f5a356afd192 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 1 FOLLOWER]: Refusing update from remote peer f5b4b9cf1f114d799efc1e159dcfe7ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:13:24.885882 21399 consensus_queue.cc:1035] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:13:24.886811 21004 raft_consensus.cc:1261] T a4cfabd8f504414bbe01f5a356afd192 P 4cab97073ee041138f8cbbab9cb7ac02 [term 1 FOLLOWER]: Refusing update from remote peer f5b4b9cf1f114d799efc1e159dcfe7ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:13:24.887393 21399 consensus_queue.cc:1035] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:13:24.947477 20458 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:13:24.947728 20458 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46916:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:13:24.950742 20458 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f: Sending DeleteTablet for 3 replicas of tablet a4cfabd8f504414bbe01f5a356afd192
I20260811 00:13:24.951690 20959 tablet_service.cc:1514] Processing DeleteTablet for tablet a4cfabd8f504414bbe01f5a356afd192 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:24 UTC) from {username='oryx', principal='oryx/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:35484
I20260811 00:13:24.951848 21221 tablet_service.cc:1514] Processing DeleteTablet for tablet a4cfabd8f504414bbe01f5a356afd192 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:24 UTC) from {username='oryx', principal='oryx/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:46648
I20260811 00:13:24.952513 20711 tablet_service.cc:1514] Processing DeleteTablet for tablet a4cfabd8f504414bbe01f5a356afd192 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:24 UTC) from {username='oryx', principal='oryx/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:59750
I20260811 00:13:24.969393 21506 tablet_replica.cc:330] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba: stopping tablet replica
I20260811 00:13:24.969666 21506 raft_consensus.cc:2229] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:13:24.970430 21506 raft_consensus.cc:2258] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:24.971895 21496 tablet_replica.cc:330] T a4cfabd8f504414bbe01f5a356afd192 P 4cab97073ee041138f8cbbab9cb7ac02: stopping tablet replica
I20260811 00:13:24.971886 21506 ts_tablet_manager.cc:1905] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:24.972344 21496 raft_consensus.cc:2229] T a4cfabd8f504414bbe01f5a356afd192 P 4cab97073ee041138f8cbbab9cb7ac02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:13:24.972668 21496 raft_consensus.cc:2258] T a4cfabd8f504414bbe01f5a356afd192 P 4cab97073ee041138f8cbbab9cb7ac02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:24.974086 21496 ts_tablet_manager.cc:1905] T a4cfabd8f504414bbe01f5a356afd192 P 4cab97073ee041138f8cbbab9cb7ac02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:24.981657 21499 tablet_replica.cc:330] T a4cfabd8f504414bbe01f5a356afd192 P 0a4ad80f7af04c30bd177b4f3582ed64: stopping tablet replica
I20260811 00:13:24.981881 21499 raft_consensus.cc:2229] T a4cfabd8f504414bbe01f5a356afd192 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:13:24.982069 21496 ts_tablet_manager.cc:1918] T a4cfabd8f504414bbe01f5a356afd192 P 4cab97073ee041138f8cbbab9cb7ac02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:13:24.982231 21496 log.cc:1196] T a4cfabd8f504414bbe01f5a356afd192 P 4cab97073ee041138f8cbbab9cb7ac02: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/a4cfabd8f504414bbe01f5a356afd192
I20260811 00:13:24.982219 21499 raft_consensus.cc:2258] T a4cfabd8f504414bbe01f5a356afd192 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:24.982750 21496 ts_tablet_manager.cc:1939] T a4cfabd8f504414bbe01f5a356afd192 P 4cab97073ee041138f8cbbab9cb7ac02: Deleting consensus metadata
I20260811 00:13:24.983978 20435 catalog_manager.cc:4841] TS 4cab97073ee041138f8cbbab9cb7ac02 (127.7.94.2:43447): tablet a4cfabd8f504414bbe01f5a356afd192 (table test-table [id=11d770a1ef364a1e8be6dfd618854687]) successfully deleted
I20260811 00:13:24.985992 21499 ts_tablet_manager.cc:1905] T a4cfabd8f504414bbe01f5a356afd192 P 0a4ad80f7af04c30bd177b4f3582ed64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:24.987562 21506 ts_tablet_manager.cc:1918] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:13:24.987725 21506 log.cc:1196] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/a4cfabd8f504414bbe01f5a356afd192
I20260811 00:13:24.988302 21506 ts_tablet_manager.cc:1939] T a4cfabd8f504414bbe01f5a356afd192 P f5b4b9cf1f114d799efc1e159dcfe7ba: Deleting consensus metadata
I20260811 00:13:24.989548 20433 catalog_manager.cc:4841] TS f5b4b9cf1f114d799efc1e159dcfe7ba (127.7.94.1:37117): tablet a4cfabd8f504414bbe01f5a356afd192 (table test-table [id=11d770a1ef364a1e8be6dfd618854687]) successfully deleted
I20260811 00:13:24.997188 21499 ts_tablet_manager.cc:1918] T a4cfabd8f504414bbe01f5a356afd192 P 0a4ad80f7af04c30bd177b4f3582ed64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:13:24.997433 21499 log.cc:1196] T a4cfabd8f504414bbe01f5a356afd192 P 0a4ad80f7af04c30bd177b4f3582ed64: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/a4cfabd8f504414bbe01f5a356afd192
I20260811 00:13:24.998099 21499 ts_tablet_manager.cc:1939] T a4cfabd8f504414bbe01f5a356afd192 P 0a4ad80f7af04c30bd177b4f3582ed64: Deleting consensus metadata
I20260811 00:13:24.999469 20434 catalog_manager.cc:4841] TS 0a4ad80f7af04c30bd177b4f3582ed64 (127.7.94.3:45029): tablet a4cfabd8f504414bbe01f5a356afd192 (table test-table [id=11d770a1ef364a1e8be6dfd618854687]) successfully deleted
Aug 11 00:13:25 dist-test-slave-8hhm krb5kdc[20383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 00:13:25 dist-test-slave-8hhm krb5kdc[20383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:25 dist-test-slave-8hhm krb5kdc[20383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 00:13:25.046675 21457 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:25.035411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46930 (local address 127.7.94.62:43779)
0811 00:13:25.035636 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:25.035644 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:25.036000 (+ 356us) server_negotiation.cc:407] Connection header received
0811 00:13:25.036174 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:25.036181 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:25.036271 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:25.036398 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:25.037732 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.039003 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.040336 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.041084 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.041275 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:25.046291 (+ 5016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46930: BlockingRecv error: recv got EOF from 127.0.0.1:46930 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 00:13:25 dist-test-slave-8hhm krb5kdc[20383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM
I20260811 00:13:25.066267 21457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:25.049810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46932 (local address 127.7.94.62:43779)
0811 00:13:25.050004 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:25.050010 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:25.050570 (+ 560us) server_negotiation.cc:407] Connection header received
0811 00:13:25.050760 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:25.050763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:25.050825 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:25.050906 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:25.052625 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.053423 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.055476 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.055897 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.055992 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:25.060903 (+ 4911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:25.060908 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:25.060911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:25.060945 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:25.063604 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:25.064239 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:25.064245 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:25.064250 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:25.064375 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:25.064756 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:25.064761 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:25.064764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:25.065030 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:25.065347 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:25.065676 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 00:13:25.065921 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 00:13:25.069789 20458 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:13:25.070417 20458 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:13:25.077996 20711 tablet_service.cc:1467] Processing CreateTablet for tablet 724a902abeaf4167852aa30af4f0b357 (DEFAULT_TABLE table=test-table [id=6ddd0ec7502343b19c24e7aa71b0d3dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:25.078509 20711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 724a902abeaf4167852aa30af4f0b357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:25.081894 21221 tablet_service.cc:1467] Processing CreateTablet for tablet 724a902abeaf4167852aa30af4f0b357 (DEFAULT_TABLE table=test-table [id=6ddd0ec7502343b19c24e7aa71b0d3dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:25.082368 21221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 724a902abeaf4167852aa30af4f0b357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:25.082273 20959 tablet_service.cc:1467] Processing CreateTablet for tablet 724a902abeaf4167852aa30af4f0b357 (DEFAULT_TABLE table=test-table [id=6ddd0ec7502343b19c24e7aa71b0d3dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:25.082716 20959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 724a902abeaf4167852aa30af4f0b357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:25.087074 21383 tablet_bootstrap.cc:492] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64: Bootstrap starting.
I20260811 00:13:25.088606 21383 tablet_bootstrap.cc:654] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:25.090337 21383 tablet_bootstrap.cc:492] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64: No bootstrap required, opened a new log
I20260811 00:13:25.090505 21383 ts_tablet_manager.cc:1397] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260811 00:13:25.091385 21383 raft_consensus.cc:348] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:25.091663 21383 raft_consensus.cc:374] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:25.091764 21382 tablet_bootstrap.cc:492] T 724a902abeaf4167852aa30af4f0b357 P f5b4b9cf1f114d799efc1e159dcfe7ba: Bootstrap starting.
I20260811 00:13:25.091789 21383 raft_consensus.cc:729] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a4ad80f7af04c30bd177b4f3582ed64, State: Initialized, Role: FOLLOWER
I20260811 00:13:25.092062 21383 consensus_queue.cc:260] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:25.092535 21383 ts_tablet_manager.cc:1428] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 00:13:25.093568 21382 tablet_bootstrap.cc:654] T 724a902abeaf4167852aa30af4f0b357 P f5b4b9cf1f114d799efc1e159dcfe7ba: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:25.094568 21392 tablet_bootstrap.cc:492] T 724a902abeaf4167852aa30af4f0b357 P 4cab97073ee041138f8cbbab9cb7ac02: Bootstrap starting.
I20260811 00:13:25.096437 21392 tablet_bootstrap.cc:654] T 724a902abeaf4167852aa30af4f0b357 P 4cab97073ee041138f8cbbab9cb7ac02: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:25.101083 21392 tablet_bootstrap.cc:492] T 724a902abeaf4167852aa30af4f0b357 P 4cab97073ee041138f8cbbab9cb7ac02: No bootstrap required, opened a new log
I20260811 00:13:25.101296 21392 ts_tablet_manager.cc:1397] T 724a902abeaf4167852aa30af4f0b357 P 4cab97073ee041138f8cbbab9cb7ac02: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 00:13:25.102187 21392 raft_consensus.cc:348] T 724a902abeaf4167852aa30af4f0b357 P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:25.102458 21392 raft_consensus.cc:374] T 724a902abeaf4167852aa30af4f0b357 P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:25.102563 21392 raft_consensus.cc:729] T 724a902abeaf4167852aa30af4f0b357 P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cab97073ee041138f8cbbab9cb7ac02, State: Initialized, Role: FOLLOWER
I20260811 00:13:25.102854 21392 consensus_queue.cc:260] T 724a902abeaf4167852aa30af4f0b357 P 4cab97073ee041138f8cbbab9cb7ac02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:25.103328 21382 tablet_bootstrap.cc:492] T 724a902abeaf4167852aa30af4f0b357 P f5b4b9cf1f114d799efc1e159dcfe7ba: No bootstrap required, opened a new log
I20260811 00:13:25.103471 21392 ts_tablet_manager.cc:1428] T 724a902abeaf4167852aa30af4f0b357 P 4cab97073ee041138f8cbbab9cb7ac02: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 00:13:25.103488 21382 ts_tablet_manager.cc:1397] T 724a902abeaf4167852aa30af4f0b357 P f5b4b9cf1f114d799efc1e159dcfe7ba: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260811 00:13:25.104290 21382 raft_consensus.cc:348] T 724a902abeaf4167852aa30af4f0b357 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:25.104559 21382 raft_consensus.cc:374] T 724a902abeaf4167852aa30af4f0b357 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:25.104697 21382 raft_consensus.cc:729] T 724a902abeaf4167852aa30af4f0b357 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5b4b9cf1f114d799efc1e159dcfe7ba, State: Initialized, Role: FOLLOWER
I20260811 00:13:25.105010 21382 consensus_queue.cc:260] T 724a902abeaf4167852aa30af4f0b357 P f5b4b9cf1f114d799efc1e159dcfe7ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:25.105993 21382 ts_tablet_manager.cc:1428] T 724a902abeaf4167852aa30af4f0b357 P f5b4b9cf1f114d799efc1e159dcfe7ba: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 00:13:25.198254 21401 raft_consensus.cc:482] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:13:25.198562 21401 raft_consensus.cc:504] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:25.200695 21401 leader_election.cc:288] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5b4b9cf1f114d799efc1e159dcfe7ba (127.7.94.1:37117), 4cab97073ee041138f8cbbab9cb7ac02 (127.7.94.2:43447)
I20260811 00:13:25.211124 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:25.201275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:37415 (local address 127.7.94.1:37117)
0811 00:13:25.201551 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:25.201564 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:25.202143 (+ 579us) server_negotiation.cc:407] Connection header received
0811 00:13:25.202484 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:25.202493 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:25.202600 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:25.202674 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:25.203554 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.204963 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.208215 (+ 3252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.209541 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.209640 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:25.209785 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:25.210516 (+ 731us) server_negotiation.cc:299] Negotiation successful
0811 00:13:25.210712 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 00:13:25.212503 21537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:25.200984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:37117 (local address 127.7.94.3:37415)
0811 00:13:25.201994 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:25.202019 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:25.202299 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:25.202928 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:25.202935 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:25.202967 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:25.203314 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:25.203329 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.205247 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.205254 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:25.207979 (+ 2725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:25.207999 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.209988 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.209997 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:25.210276 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:25.210330 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 00:13:25.211718 (+ 1388us) client_negotiation.cc:240] Negotiation successful
0811 00:13:25.211961 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":881,"threads_started":1}
I20260811 00:13:25.214339 20753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "724a902abeaf4167852aa30af4f0b357" candidate_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" is_pre_election: true
I20260811 00:13:25.214666 20753 raft_consensus.cc:2454] T 724a902abeaf4167852aa30af4f0b357 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a4ad80f7af04c30bd177b4f3582ed64 in term 0.
I20260811 00:13:25.215255 21129 leader_election.cc:302] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a4ad80f7af04c30bd177b4f3582ed64, f5b4b9cf1f114d799efc1e159dcfe7ba; no voters:
I20260811 00:13:25.215727 21401 raft_consensus.cc:2781] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:13:25.215905 21401 raft_consensus.cc:482] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:13:25.216044 21401 raft_consensus.cc:3037] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:25.218690 21401 raft_consensus.cc:504] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:25.219525 21401 leader_election.cc:288] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [CANDIDATE]: Term 1 election: Requested vote from peers f5b4b9cf1f114d799efc1e159dcfe7ba (127.7.94.1:37117), 4cab97073ee041138f8cbbab9cb7ac02 (127.7.94.2:43447)
I20260811 00:13:25.219919 20753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "724a902abeaf4167852aa30af4f0b357" candidate_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba"
I20260811 00:13:25.220186 20753 raft_consensus.cc:3037] T 724a902abeaf4167852aa30af4f0b357 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:25.220768 21536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:25.201149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:33021 (local address 127.7.94.2:43447)
0811 00:13:25.214083 (+ 12934us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:25.214089 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:25.214122 (+ 33us) server_negotiation.cc:407] Connection header received
0811 00:13:25.214197 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:25.214200 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:25.214258 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:25.214370 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:25.214745 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.215652 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.218397 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.219742 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.219879 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:25.219982 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:25.220299 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 00:13:25.220437 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12790,"thread_start_us":12674,"threads_started":1}
I20260811 00:13:25.220793 21537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:25.200984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:43447 (local address 127.7.94.3:33021)
0811 00:13:25.213017 (+ 12033us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:25.213042 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:25.213198 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:25.214391 (+ 1193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:25.214396 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:25.214423 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:25.214580 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:25.214592 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.215807 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.215815 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:25.218148 (+ 2333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:25.218184 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.219911 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.219917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:25.220106 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:25.220150 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 00:13:25.220270 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 00:13:25.220404 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11865,"mutex_wait_us":275,"thread_start_us":488,"threads_started":1}
I20260811 00:13:25.223774 20753 raft_consensus.cc:2454] T 724a902abeaf4167852aa30af4f0b357 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a4ad80f7af04c30bd177b4f3582ed64 in term 1.
I20260811 00:13:25.224197 21129 leader_election.cc:302] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a4ad80f7af04c30bd177b4f3582ed64, f5b4b9cf1f114d799efc1e159dcfe7ba; no voters:
I20260811 00:13:25.224496 21401 raft_consensus.cc:2781] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:13:25.224725 21004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "724a902abeaf4167852aa30af4f0b357" candidate_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cab97073ee041138f8cbbab9cb7ac02" is_pre_election: true
I20260811 00:13:25.224802 20995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "724a902abeaf4167852aa30af4f0b357" candidate_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cab97073ee041138f8cbbab9cb7ac02"
I20260811 00:13:25.224999 21004 raft_consensus.cc:2454] T 724a902abeaf4167852aa30af4f0b357 P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a4ad80f7af04c30bd177b4f3582ed64 in term 0.
I20260811 00:13:25.225741 21401 raft_consensus.cc:686] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 1 LEADER]: Becoming Leader. State: Replica: 0a4ad80f7af04c30bd177b4f3582ed64, State: Running, Role: LEADER
I20260811 00:13:25.226292 21401 consensus_queue.cc:237] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:25.230316 20453 catalog_manager.cc:5495] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a4ad80f7af04c30bd177b4f3582ed64 (127.7.94.3). New cstate: current_term: 1 leader_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } health_report { overall_health: HEALTHY } } }
I20260811 00:13:25.252383 21549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:25.244935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46652 (local address 127.7.94.3:45029)
0811 00:13:25.245751 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:25.245755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:25.245774 (+ 19us) server_negotiation.cc:407] Connection header received
0811 00:13:25.245851 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:25.245854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:25.245927 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:25.246065 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:13:25.246654 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.247750 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.249929 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.250333 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.250426 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:25.251045 (+ 619us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:13:25.251624 (+ 579us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:13:25.251742 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:25.251942 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 00:13:25.252012 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":563,"threads_started":1}
I20260811 00:13:25.256464 20995 raft_consensus.cc:3037] T 724a902abeaf4167852aa30af4f0b357 P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:25.256646 20753 raft_consensus.cc:1261] T 724a902abeaf4167852aa30af4f0b357 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 1 FOLLOWER]: Refusing update from remote peer 0a4ad80f7af04c30bd177b4f3582ed64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:13:25.257467 21538 consensus_queue.cc:1035] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:13:25.259624 20995 raft_consensus.cc:1261] T 724a902abeaf4167852aa30af4f0b357 P 4cab97073ee041138f8cbbab9cb7ac02 [term 1 FOLLOWER]: Refusing update from remote peer 0a4ad80f7af04c30bd177b4f3582ed64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:13:25.260293 21538 consensus_queue.cc:1035] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:13:25.282920 20453 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:13:25.283172 20453 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:13:25.286671 20453 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f: Sending DeleteTablet for 3 replicas of tablet 724a902abeaf4167852aa30af4f0b357
I20260811 00:13:25.287740 20711 tablet_service.cc:1514] Processing DeleteTablet for tablet 724a902abeaf4167852aa30af4f0b357 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:25 UTC) from {username='oryx', principal='oryx/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:59750
I20260811 00:13:25.287920 21221 tablet_service.cc:1514] Processing DeleteTablet for tablet 724a902abeaf4167852aa30af4f0b357 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:25 UTC) from {username='oryx', principal='oryx/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:46648
I20260811 00:13:25.288116 20959 tablet_service.cc:1514] Processing DeleteTablet for tablet 724a902abeaf4167852aa30af4f0b357 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:25 UTC) from {username='oryx', principal='oryx/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:35484
I20260811 00:13:25.288208 21499 tablet_replica.cc:330] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64: stopping tablet replica
I20260811 00:13:25.288425 21506 tablet_replica.cc:330] T 724a902abeaf4167852aa30af4f0b357 P f5b4b9cf1f114d799efc1e159dcfe7ba: stopping tablet replica
I20260811 00:13:25.288419 21499 raft_consensus.cc:2229] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:13:25.288640 21506 raft_consensus.cc:2229] T 724a902abeaf4167852aa30af4f0b357 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:13:25.288874 21506 raft_consensus.cc:2258] T 724a902abeaf4167852aa30af4f0b357 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:25.288880 21499 raft_consensus.cc:2258] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:25.289165 21496 tablet_replica.cc:330] T 724a902abeaf4167852aa30af4f0b357 P 4cab97073ee041138f8cbbab9cb7ac02: stopping tablet replica
I20260811 00:13:25.289383 21496 raft_consensus.cc:2229] T 724a902abeaf4167852aa30af4f0b357 P 4cab97073ee041138f8cbbab9cb7ac02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:13:25.289618 21496 raft_consensus.cc:2258] T 724a902abeaf4167852aa30af4f0b357 P 4cab97073ee041138f8cbbab9cb7ac02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:25.289983 21499 ts_tablet_manager.cc:1905] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:25.292821 21506 ts_tablet_manager.cc:1905] T 724a902abeaf4167852aa30af4f0b357 P f5b4b9cf1f114d799efc1e159dcfe7ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:25.293875 21496 ts_tablet_manager.cc:1905] T 724a902abeaf4167852aa30af4f0b357 P 4cab97073ee041138f8cbbab9cb7ac02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:25.300765 20453 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:13:25.301549 20453 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:13:25.306124 21496 ts_tablet_manager.cc:1918] T 724a902abeaf4167852aa30af4f0b357 P 4cab97073ee041138f8cbbab9cb7ac02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:13:25.306308 21496 log.cc:1196] T 724a902abeaf4167852aa30af4f0b357 P 4cab97073ee041138f8cbbab9cb7ac02: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/724a902abeaf4167852aa30af4f0b357
I20260811 00:13:25.307056 21496 ts_tablet_manager.cc:1939] T 724a902abeaf4167852aa30af4f0b357 P 4cab97073ee041138f8cbbab9cb7ac02: Deleting consensus metadata
I20260811 00:13:25.308231 20435 catalog_manager.cc:4841] TS 4cab97073ee041138f8cbbab9cb7ac02 (127.7.94.2:43447): tablet 724a902abeaf4167852aa30af4f0b357 (table test-table [id=6ddd0ec7502343b19c24e7aa71b0d3dc]) successfully deleted
I20260811 00:13:25.310158 20711 tablet_service.cc:1467] Processing CreateTablet for tablet 75b5502b73174a299d0b2dc1f6086e80 (DEFAULT_TABLE table=test-table [id=91f245b3bdbd44f0bdb920556eb36179]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:25.310541 21221 tablet_service.cc:1467] Processing CreateTablet for tablet 75b5502b73174a299d0b2dc1f6086e80 (DEFAULT_TABLE table=test-table [id=91f245b3bdbd44f0bdb920556eb36179]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:25.310652 20711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75b5502b73174a299d0b2dc1f6086e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:25.313987 21506 ts_tablet_manager.cc:1918] T 724a902abeaf4167852aa30af4f0b357 P f5b4b9cf1f114d799efc1e159dcfe7ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:13:25.314232 21506 log.cc:1196] T 724a902abeaf4167852aa30af4f0b357 P f5b4b9cf1f114d799efc1e159dcfe7ba: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/724a902abeaf4167852aa30af4f0b357
I20260811 00:13:25.314621 21221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75b5502b73174a299d0b2dc1f6086e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:25.317584 20959 tablet_service.cc:1467] Processing CreateTablet for tablet 75b5502b73174a299d0b2dc1f6086e80 (DEFAULT_TABLE table=test-table [id=91f245b3bdbd44f0bdb920556eb36179]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:25.317973 20959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75b5502b73174a299d0b2dc1f6086e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:25.321960 21383 tablet_bootstrap.cc:492] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64: Bootstrap starting.
I20260811 00:13:25.322269 21499 ts_tablet_manager.cc:1918] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:13:25.322818 21506 ts_tablet_manager.cc:1939] T 724a902abeaf4167852aa30af4f0b357 P f5b4b9cf1f114d799efc1e159dcfe7ba: Deleting consensus metadata
I20260811 00:13:25.323053 21499 log.cc:1196] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/724a902abeaf4167852aa30af4f0b357
I20260811 00:13:25.323647 21499 ts_tablet_manager.cc:1939] T 724a902abeaf4167852aa30af4f0b357 P 0a4ad80f7af04c30bd177b4f3582ed64: Deleting consensus metadata
I20260811 00:13:25.324878 20433 catalog_manager.cc:4841] TS f5b4b9cf1f114d799efc1e159dcfe7ba (127.7.94.1:37117): tablet 724a902abeaf4167852aa30af4f0b357 (table test-table [id=6ddd0ec7502343b19c24e7aa71b0d3dc]) successfully deleted
I20260811 00:13:25.323982 21383 tablet_bootstrap.cc:654] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:25.330389 20434 catalog_manager.cc:4841] TS 0a4ad80f7af04c30bd177b4f3582ed64 (127.7.94.3:45029): tablet 724a902abeaf4167852aa30af4f0b357 (table test-table [id=6ddd0ec7502343b19c24e7aa71b0d3dc]) successfully deleted
I20260811 00:13:25.331899 21392 tablet_bootstrap.cc:492] T 75b5502b73174a299d0b2dc1f6086e80 P 4cab97073ee041138f8cbbab9cb7ac02: Bootstrap starting.
I20260811 00:13:25.332640 21382 tablet_bootstrap.cc:492] T 75b5502b73174a299d0b2dc1f6086e80 P f5b4b9cf1f114d799efc1e159dcfe7ba: Bootstrap starting.
I20260811 00:13:25.333678 21392 tablet_bootstrap.cc:654] T 75b5502b73174a299d0b2dc1f6086e80 P 4cab97073ee041138f8cbbab9cb7ac02: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:25.334514 21382 tablet_bootstrap.cc:654] T 75b5502b73174a299d0b2dc1f6086e80 P f5b4b9cf1f114d799efc1e159dcfe7ba: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:25.347177 21382 tablet_bootstrap.cc:492] T 75b5502b73174a299d0b2dc1f6086e80 P f5b4b9cf1f114d799efc1e159dcfe7ba: No bootstrap required, opened a new log
I20260811 00:13:25.347558 21382 ts_tablet_manager.cc:1397] T 75b5502b73174a299d0b2dc1f6086e80 P f5b4b9cf1f114d799efc1e159dcfe7ba: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.006s
I20260811 00:13:25.348287 21383 tablet_bootstrap.cc:492] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64: No bootstrap required, opened a new log
I20260811 00:13:25.348676 21382 raft_consensus.cc:348] T 75b5502b73174a299d0b2dc1f6086e80 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:25.349086 21382 raft_consensus.cc:374] T 75b5502b73174a299d0b2dc1f6086e80 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:25.349545 21382 raft_consensus.cc:729] T 75b5502b73174a299d0b2dc1f6086e80 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5b4b9cf1f114d799efc1e159dcfe7ba, State: Initialized, Role: FOLLOWER
I20260811 00:13:25.349965 21382 consensus_queue.cc:260] T 75b5502b73174a299d0b2dc1f6086e80 P f5b4b9cf1f114d799efc1e159dcfe7ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:25.350137 21383 ts_tablet_manager.cc:1397] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64: Time spent bootstrapping tablet: real 0.028s user 0.006s sys 0.006s
I20260811 00:13:25.350972 21383 raft_consensus.cc:348] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:25.351269 21383 raft_consensus.cc:374] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:25.351368 21383 raft_consensus.cc:729] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a4ad80f7af04c30bd177b4f3582ed64, State: Initialized, Role: FOLLOWER
I20260811 00:13:25.351614 21383 consensus_queue.cc:260] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:25.352653 21392 tablet_bootstrap.cc:492] T 75b5502b73174a299d0b2dc1f6086e80 P 4cab97073ee041138f8cbbab9cb7ac02: No bootstrap required, opened a new log
I20260811 00:13:25.352833 21392 ts_tablet_manager.cc:1397] T 75b5502b73174a299d0b2dc1f6086e80 P 4cab97073ee041138f8cbbab9cb7ac02: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.001s
I20260811 00:13:25.353542 21382 ts_tablet_manager.cc:1428] T 75b5502b73174a299d0b2dc1f6086e80 P f5b4b9cf1f114d799efc1e159dcfe7ba: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260811 00:13:25.353682 21392 raft_consensus.cc:348] T 75b5502b73174a299d0b2dc1f6086e80 P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:25.353941 21392 raft_consensus.cc:374] T 75b5502b73174a299d0b2dc1f6086e80 P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:25.354018 21392 raft_consensus.cc:729] T 75b5502b73174a299d0b2dc1f6086e80 P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cab97073ee041138f8cbbab9cb7ac02, State: Initialized, Role: FOLLOWER
I20260811 00:13:25.354257 21392 consensus_queue.cc:260] T 75b5502b73174a299d0b2dc1f6086e80 P 4cab97073ee041138f8cbbab9cb7ac02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:25.355973 21392 ts_tablet_manager.cc:1428] T 75b5502b73174a299d0b2dc1f6086e80 P 4cab97073ee041138f8cbbab9cb7ac02: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 00:13:25.357332 21383 ts_tablet_manager.cc:1428] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260811 00:13:25.581514 21553 raft_consensus.cc:482] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:13:25.581830 21553 raft_consensus.cc:504] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:25.582757 21553 leader_election.cc:288] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5b4b9cf1f114d799efc1e159dcfe7ba (127.7.94.1:37117), 4cab97073ee041138f8cbbab9cb7ac02 (127.7.94.2:43447)
I20260811 00:13:25.583425 20753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75b5502b73174a299d0b2dc1f6086e80" candidate_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" is_pre_election: true
I20260811 00:13:25.583705 20753 raft_consensus.cc:2454] T 75b5502b73174a299d0b2dc1f6086e80 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a4ad80f7af04c30bd177b4f3582ed64 in term 0.
I20260811 00:13:25.583577 20995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75b5502b73174a299d0b2dc1f6086e80" candidate_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cab97073ee041138f8cbbab9cb7ac02" is_pre_election: true
I20260811 00:13:25.583917 20995 raft_consensus.cc:2454] T 75b5502b73174a299d0b2dc1f6086e80 P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a4ad80f7af04c30bd177b4f3582ed64 in term 0.
I20260811 00:13:25.584276 21129 leader_election.cc:302] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a4ad80f7af04c30bd177b4f3582ed64, f5b4b9cf1f114d799efc1e159dcfe7ba; no voters:
I20260811 00:13:25.584692 21553 raft_consensus.cc:2781] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:13:25.584826 21553 raft_consensus.cc:482] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:13:25.584937 21553 raft_consensus.cc:3037] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:25.588750 21553 raft_consensus.cc:504] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:25.589960 20995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75b5502b73174a299d0b2dc1f6086e80" candidate_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cab97073ee041138f8cbbab9cb7ac02"
I20260811 00:13:25.589993 20753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75b5502b73174a299d0b2dc1f6086e80" candidate_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba"
I20260811 00:13:25.590205 20995 raft_consensus.cc:3037] T 75b5502b73174a299d0b2dc1f6086e80 P 4cab97073ee041138f8cbbab9cb7ac02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:25.590226 20753 raft_consensus.cc:3037] T 75b5502b73174a299d0b2dc1f6086e80 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:25.593571 20995 raft_consensus.cc:2454] T 75b5502b73174a299d0b2dc1f6086e80 P 4cab97073ee041138f8cbbab9cb7ac02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a4ad80f7af04c30bd177b4f3582ed64 in term 1.
I20260811 00:13:25.593819 20753 raft_consensus.cc:2454] T 75b5502b73174a299d0b2dc1f6086e80 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a4ad80f7af04c30bd177b4f3582ed64 in term 1.
I20260811 00:13:25.594066 21133 leader_election.cc:302] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a4ad80f7af04c30bd177b4f3582ed64, 4cab97073ee041138f8cbbab9cb7ac02; no voters:
I20260811 00:13:25.594410 21555 raft_consensus.cc:2781] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:13:25.594507 21553 leader_election.cc:288] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [CANDIDATE]: Term 1 election: Requested vote from peers f5b4b9cf1f114d799efc1e159dcfe7ba (127.7.94.1:37117), 4cab97073ee041138f8cbbab9cb7ac02 (127.7.94.2:43447)
I20260811 00:13:25.594640 21555 raft_consensus.cc:686] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 1 LEADER]: Becoming Leader. State: Replica: 0a4ad80f7af04c30bd177b4f3582ed64, State: Running, Role: LEADER
I20260811 00:13:25.595060 21555 consensus_queue.cc:237] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } }
I20260811 00:13:25.598425 20453 catalog_manager.cc:5495] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a4ad80f7af04c30bd177b4f3582ed64 (127.7.94.3). New cstate: current_term: 1 leader_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a4ad80f7af04c30bd177b4f3582ed64" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 45029 } health_report { overall_health: HEALTHY } } }
I20260811 00:13:25.688179 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:25.670413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:51909 (local address 127.7.94.1:37117)
0811 00:13:25.670628 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:25.670639 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:25.680272 (+ 9633us) server_negotiation.cc:407] Connection header received
0811 00:13:25.680886 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:25.680894 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:25.681000 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:25.681079 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:13:25.682037 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.683173 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.685528 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.686335 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.686440 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:25.687197 (+ 757us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:13:25.687346 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:13:25.687432 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:25.687699 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 00:13:25.687798 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:13:25.689247 21576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:25.668791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:37117 (local address 127.7.94.3:51909)
0811 00:13:25.680136 (+ 11345us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:25.680165 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:25.680673 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:25.681334 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:25.681338 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:25.681351 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:13:25.681801 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:25.681816 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.683384 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.683392 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:25.685335 (+ 1943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:25.685351 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.686696 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.686702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:25.686881 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:25.686974 (+ 93us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:13:25.687510 (+ 536us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:13:25.687528 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 00:13:25.688691 (+ 1163us) client_negotiation.cc:240] Negotiation successful
0811 00:13:25.688840 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11112,"thread_start_us":11503,"threads_started":1}
I20260811 00:13:25.700790 20753 raft_consensus.cc:1261] T 75b5502b73174a299d0b2dc1f6086e80 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 1 FOLLOWER]: Refusing update from remote peer 0a4ad80f7af04c30bd177b4f3582ed64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:13:25.701501 21401 consensus_queue.cc:1035] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5b4b9cf1f114d799efc1e159dcfe7ba" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:13:25.702593 20995 raft_consensus.cc:1261] T 75b5502b73174a299d0b2dc1f6086e80 P 4cab97073ee041138f8cbbab9cb7ac02 [term 1 FOLLOWER]: Refusing update from remote peer 0a4ad80f7af04c30bd177b4f3582ed64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:13:25.703174 21401 consensus_queue.cc:1035] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cab97073ee041138f8cbbab9cb7ac02" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:13:25.805001 21587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:25.782104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:45029 (local address 127.7.94.1:51821)
0811 00:13:25.797649 (+ 15545us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:25.797669 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:25.797822 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:25.798269 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:25.798276 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:25.798288 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 00:13:25.798762 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:25.798780 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.800468 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.800473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:25.802261 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:25.802277 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.803444 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.803451 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:25.803669 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:25.803721 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 00:13:25.804329 (+ 608us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 00:13:25.804341 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 00:13:25.804425 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 00:13:25.804556 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15003,"thread_start_us":15127,"threads_started":1}
I20260811 00:13:25.805779 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:25.782091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:51821 (local address 127.7.94.3:45029)
0811 00:13:25.796806 (+ 14715us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:25.796814 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:25.797773 (+ 959us) server_negotiation.cc:407] Connection header received
0811 00:13:25.797993 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:25.798000 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:25.798122 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:25.798236 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:13:25.798975 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.800290 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.802464 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.803153 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.803282 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:25.803990 (+ 708us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:13:25.804165 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:13:25.804269 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:25.805297 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0811 00:13:25.805380 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14570,"thread_start_us":14736,"threads_started":1}
I20260811 00:13:25.839062 20458 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:13:25.839334 20458 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:13:25.842418 20458 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a2b21918e3524c69a10ca87835f94a2f: Sending DeleteTablet for 3 replicas of tablet 75b5502b73174a299d0b2dc1f6086e80
I20260811 00:13:25.843596 21221 tablet_service.cc:1514] Processing DeleteTablet for tablet 75b5502b73174a299d0b2dc1f6086e80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:25 UTC) from {username='oryx', principal='oryx/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:46648
I20260811 00:13:25.843813 20711 tablet_service.cc:1514] Processing DeleteTablet for tablet 75b5502b73174a299d0b2dc1f6086e80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:25 UTC) from {username='oryx', principal='oryx/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:59750
I20260811 00:13:25.844056 20959 tablet_service.cc:1514] Processing DeleteTablet for tablet 75b5502b73174a299d0b2dc1f6086e80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:25 UTC) from {username='oryx', principal='oryx/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:35484
I20260811 00:13:25.845094 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 20584
I20260811 00:13:25.868460 21593 tablet_replica.cc:330] T 75b5502b73174a299d0b2dc1f6086e80 P 4cab97073ee041138f8cbbab9cb7ac02: stopping tablet replica
I20260811 00:13:25.868719 21593 raft_consensus.cc:2229] T 75b5502b73174a299d0b2dc1f6086e80 P 4cab97073ee041138f8cbbab9cb7ac02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:13:25.868953 21593 raft_consensus.cc:2258] T 75b5502b73174a299d0b2dc1f6086e80 P 4cab97073ee041138f8cbbab9cb7ac02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:25.870117 21593 ts_tablet_manager.cc:1905] T 75b5502b73174a299d0b2dc1f6086e80 P 4cab97073ee041138f8cbbab9cb7ac02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:25.872519 21590 tablet_replica.cc:330] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64: stopping tablet replica
I20260811 00:13:25.873476 21590 raft_consensus.cc:2229] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:13:25.874032 21591 tablet_replica.cc:330] T 75b5502b73174a299d0b2dc1f6086e80 P f5b4b9cf1f114d799efc1e159dcfe7ba: stopping tablet replica
I20260811 00:13:25.874233 21591 raft_consensus.cc:2229] T 75b5502b73174a299d0b2dc1f6086e80 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:13:25.874522 21591 raft_consensus.cc:2258] T 75b5502b73174a299d0b2dc1f6086e80 P f5b4b9cf1f114d799efc1e159dcfe7ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:25.874585 21590 raft_consensus.cc:2258] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:25.876528 21590 ts_tablet_manager.cc:1905] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:25.880445 21593 ts_tablet_manager.cc:1918] T 75b5502b73174a299d0b2dc1f6086e80 P 4cab97073ee041138f8cbbab9cb7ac02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:13:25.880867 21591 ts_tablet_manager.cc:1905] T 75b5502b73174a299d0b2dc1f6086e80 P f5b4b9cf1f114d799efc1e159dcfe7ba: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 00:13:25 dist-test-slave-8hhm krb5kdc[20383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.94.1@KRBTEST.COM
I20260811 00:13:25.888296 21591 ts_tablet_manager.cc:1918] T 75b5502b73174a299d0b2dc1f6086e80 P f5b4b9cf1f114d799efc1e159dcfe7ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:13:25.888490 21591 log.cc:1196] T 75b5502b73174a299d0b2dc1f6086e80 P f5b4b9cf1f114d799efc1e159dcfe7ba: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/75b5502b73174a299d0b2dc1f6086e80
I20260811 00:13:25.889065 21591 ts_tablet_manager.cc:1939] T 75b5502b73174a299d0b2dc1f6086e80 P f5b4b9cf1f114d799efc1e159dcfe7ba: Deleting consensus metadata
I20260811 00:13:25.880596 21593 log.cc:1196] T 75b5502b73174a299d0b2dc1f6086e80 P 4cab97073ee041138f8cbbab9cb7ac02: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/75b5502b73174a299d0b2dc1f6086e80
I20260811 00:13:25.890714 21593 ts_tablet_manager.cc:1939] T 75b5502b73174a299d0b2dc1f6086e80 P 4cab97073ee041138f8cbbab9cb7ac02: Deleting consensus metadata
I20260811 00:13:25.891462 20433 catalog_manager.cc:4841] TS f5b4b9cf1f114d799efc1e159dcfe7ba (127.7.94.1:37117): tablet 75b5502b73174a299d0b2dc1f6086e80 (table test-table [id=91f245b3bdbd44f0bdb920556eb36179]) successfully deleted
I20260811 00:13:25.892196 20435 catalog_manager.cc:4841] TS 4cab97073ee041138f8cbbab9cb7ac02 (127.7.94.2:43447): tablet 75b5502b73174a299d0b2dc1f6086e80 (table test-table [id=91f245b3bdbd44f0bdb920556eb36179]) successfully deleted
I20260811 00:13:25.894546 21590 ts_tablet_manager.cc:1918] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 00:13:25.894727 21590 log.cc:1196] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/75b5502b73174a299d0b2dc1f6086e80
I20260811 00:13:25.895145 21590 ts_tablet_manager.cc:1939] T 75b5502b73174a299d0b2dc1f6086e80 P 0a4ad80f7af04c30bd177b4f3582ed64: Deleting consensus metadata
I20260811 00:13:25.896148 20434 catalog_manager.cc:4841] TS 0a4ad80f7af04c30bd177b4f3582ed64 (127.7.94.3:45029): tablet 75b5502b73174a299d0b2dc1f6086e80 (table test-table [id=91f245b3bdbd44f0bdb920556eb36179]) successfully deleted
I20260811 00:13:25.896914 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:25.846929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59782 (local address 127.7.94.1:37117)
0811 00:13:25.847132 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:25.847141 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:25.868309 (+ 21168us) server_negotiation.cc:407] Connection header received
0811 00:13:25.868780 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:25.868787 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:25.868878 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:25.868958 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:25.870994 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.872021 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.876723 (+ 4702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:25.877436 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:25.877544 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:25.888668 (+ 11124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:25.888694 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:25.888718 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:25.888771 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:25.893600 (+ 4829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:25.894323 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:25.894334 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:25.894342 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:25.894465 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:25.895021 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:25.895032 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:25.895035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:25.895520 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:25.895888 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:25.896303 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 00:13:25.896554 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 00:13:25.897711 20818 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:26.059820 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 20584
I20260811 00:13:26.098047 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 20833
Aug 11 00:13:26 dist-test-slave-8hhm krb5kdc[20383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.94.2@KRBTEST.COM
I20260811 00:13:26.112511 21599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:26.098933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35502 (local address 127.7.94.2:43447)
0811 00:13:26.099437 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:26.099442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:26.099458 (+ 16us) server_negotiation.cc:407] Connection header received
0811 00:13:26.099513 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:26.099517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:26.099596 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:26.099687 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:26.100902 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:26.101688 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:26.102839 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:26.103261 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:26.103352 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:26.107290 (+ 3938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:26.107303 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:26.107318 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:26.107349 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:26.110086 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:26.110659 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:26.110666 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:26.110669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:26.110792 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:26.111104 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:26.111107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:26.111109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:26.111488 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:26.111712 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:26.111989 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 00:13:26.112144 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":271,"threads_started":1}
I20260811 00:13:26.113161 21076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:26.239737 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 20833
I20260811 00:13:26.270684 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 21091
Aug 11 00:13:26 dist-test-slave-8hhm krb5kdc[20383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.94.3@KRBTEST.COM
I20260811 00:13:26.285804 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:26.271500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46658 (local address 127.7.94.3:45029)
0811 00:13:26.271717 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:26.271724 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:26.271746 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:13:26.271961 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:26.271966 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:26.272062 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:26.272181 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:26.273620 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:26.274327 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:26.275559 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:26.275998 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:26.276146 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:26.280421 (+ 4275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:26.280436 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:26.280453 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:26.280486 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:26.283334 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:26.283936 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:26.283943 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:26.283946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:26.284078 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:26.284388 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:26.284391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:26.284393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:26.284761 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:26.285005 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:26.285207 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 00:13:26.285463 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 00:13:26.286415 21334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:26.411897 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 21091
I20260811 00:13:26.445191 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 20407
I20260811 00:13:26.446434 20533 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:26.561261 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 20407
2026-08-11T00:13:26Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6650 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:13:26 dist-test-slave-8hhm krb5kdc[21608](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:13:26 dist-test-slave-8hhm krb5kdc[21608](info): set up 2 sockets
Aug 11 00:13:26 dist-test-slave-8hhm krb5kdc[21608](info): commencing operation
krb5kdc: starting...
W20260811 00:13:27.910714 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:13:28 dist-test-slave-8hhm krb5kdc[21608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:13:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:13:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:13:28.157838 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:39717
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:36469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:39717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:28.479924 21633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:28.480209 21633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:28.480373 21633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:28.487251 21633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:13:28.487344 21633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:28.487387 21633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:28.487468 21633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:13:28.487543 21633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:13:28.496977 21633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:36469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:39717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:39717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:28.498252 21633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:28.499536 21633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:28.511170 21644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:28.511225 21643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:28.511253 21646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:28.512423 21633 server_base.cc:1034] running on GCE node
I20260811 00:13:28.513157 21633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:28.514511 21633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:28.515718 21633 hybrid_clock.cc:648] HybridClock initialized: now 1786407208515678 us; error 25 us; skew 500 ppm
I20260811 00:13:28.516026 21633 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T00:13:28Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2019 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:13:28 dist-test-slave-8hhm krb5kdc[21656](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:13:28 dist-test-slave-8hhm krb5kdc[21656](info): set up 2 sockets
Aug 11 00:13:28 dist-test-slave-8hhm krb5kdc[21656](info): commencing operation
krb5kdc: starting...
W20260811 00:13:29.921195 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.003s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:13:30 dist-test-slave-8hhm krb5kdc[21656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:13:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:13:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:13:30.184374 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:35503
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:35503
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:30.513156 21670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:30.513527 21670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:30.513638 21670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:30.520905 21670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:13:30.521035 21670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:30.521090 21670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:30.521157 21670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:13:30.521260 21670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:13:30.531541 21670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:35503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:35503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:30.532882 21670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:30.534368 21670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:30.546768 21688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:30.546859 21681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:30.546859 21685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:30.548113 21670 server_base.cc:1034] running on GCE node
I20260811 00:13:30.548930 21670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:30.550495 21670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:30.551777 21670 hybrid_clock.cc:648] HybridClock initialized: now 1786407210551721 us; error 37 us; skew 500 ppm
I20260811 00:13:30.552165 21670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:30 dist-test-slave-8hhm krb5kdc[21656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407210, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:30.555621 21670 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:13:30.557265 21670 webserver.cc:460] Webserver started at http://127.7.94.62:34241/ using document root <none> and password file <none>
I20260811 00:13:30.557940 21670 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:30.558061 21670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:30.558346 21670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:30.561471 21670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "890248e3001045859606028605dde43e"
format_stamp: "Formatted at 2026-08-11 00:13:30 on dist-test-slave-8hhm"
server_key: "19f612e920fbe78ce3d401da2e812f79"
server_key_iv: "7a957f253e6ec0d3507b40047c51d0c5"
server_key_version: "encryptionkey@0"
I20260811 00:13:30.562319 21670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "890248e3001045859606028605dde43e"
format_stamp: "Formatted at 2026-08-11 00:13:30 on dist-test-slave-8hhm"
server_key: "19f612e920fbe78ce3d401da2e812f79"
server_key_iv: "7a957f253e6ec0d3507b40047c51d0c5"
server_key_version: "encryptionkey@0"
I20260811 00:13:30.567595 21670 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:13:30.571151 21703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:30.572465 21670 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:13:30.572606 21670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "890248e3001045859606028605dde43e"
format_stamp: "Formatted at 2026-08-11 00:13:30 on dist-test-slave-8hhm"
server_key: "19f612e920fbe78ce3d401da2e812f79"
server_key_iv: "7a957f253e6ec0d3507b40047c51d0c5"
server_key_version: "encryptionkey@0"
I20260811 00:13:30.572777 21670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:30.595803 21670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:30.599915 21670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:30.600143 21670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:30.618091 21670 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:35503
I20260811 00:13:30.618125 21787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:35503 every 8 connection(s)
I20260811 00:13:30.619378 21670 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:13:30.623237 21788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:30.626984 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 21670
I20260811 00:13:30.627215 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:13:30.628075 7544 external_mini_cluster.cc:1428] Setting key 33dc38c30ad1cda6c9fe2bf004ab0553
I20260811 00:13:30.637347 21788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e: Bootstrap starting.
I20260811 00:13:30.641400 21788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:30.642860 21788 log.cc:824] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:13:30 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:13:30.646039 21788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e: No bootstrap required, opened a new log
I20260811 00:13:30.651911 21792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:30.630198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32968 (local address 127.7.94.62:35503)
0811 00:13:30.631018 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:30.631038 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:30.631079 (+ 41us) server_negotiation.cc:407] Connection header received
0811 00:13:30.632766 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:30.632789 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:30.633176 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:30.633556 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:30.635455 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:30.636658 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:30.638540 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:30.639191 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:30.639316 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:30.644652 (+ 5336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:30.644661 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:30.644675 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:30.644712 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:30.647840 (+ 3128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:30.648540 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:30.648547 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:30.648553 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:30.648718 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:30.649127 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:30.649130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:30.649132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:30.649689 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:30.650007 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:30.650468 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 00:13:30.650883 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":373,"threads_started":1}
I20260811 00:13:30.655494 21788 raft_consensus.cc:348] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890248e3001045859606028605dde43e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35503 } }
I20260811 00:13:30.655947 21788 raft_consensus.cc:374] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:30.656075 21788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 890248e3001045859606028605dde43e, State: Initialized, Role: FOLLOWER
I20260811 00:13:30.656652 21788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890248e3001045859606028605dde43e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35503 } }
I20260811 00:13:30.656921 21788 raft_consensus.cc:388] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:13:30.657101 21788 raft_consensus.cc:482] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:13:30.657325 21788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:30.659816 21788 raft_consensus.cc:504] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890248e3001045859606028605dde43e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35503 } }
I20260811 00:13:30.660339 21788 leader_election.cc:302] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 890248e3001045859606028605dde43e; no voters:
I20260811 00:13:30.660938 21788 leader_election.cc:288] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:13:30.661038 21796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:13:30.661933 21796 raft_consensus.cc:686] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [term 1 LEADER]: Becoming Leader. State: Replica: 890248e3001045859606028605dde43e, State: Running, Role: LEADER
I20260811 00:13:30.662497 21796 consensus_queue.cc:237] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890248e3001045859606028605dde43e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35503 } }
I20260811 00:13:30.663357 21788 sys_catalog.cc:564] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:13:30.665189 21799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 890248e3001045859606028605dde43e. Latest consensus state: current_term: 1 leader_uuid: "890248e3001045859606028605dde43e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890248e3001045859606028605dde43e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35503 } } }
I20260811 00:13:30.666020 21798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "890248e3001045859606028605dde43e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890248e3001045859606028605dde43e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35503 } } }
I20260811 00:13:30.666388 21799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:30.666486 21798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:30.667272 21809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:13:30.673269 21809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:13:30.681707 21809 catalog_manager.cc:1347] Generated new cluster ID: a3b6b7c5d24d4b04b051575aa7fccbd6
I20260811 00:13:30.681811 21809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:13:30.710234 21809 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:13:30.711549 21809 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:13:30.719578 21809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e: Generated new TSK 0
I20260811 00:13:30.720408 21809 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
I20260811 00:13:30.802971 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:35503
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:31.199507 21836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:31.199764 21836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:31.199852 21836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:31.207360 21836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:31.207486 21836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:31.207651 21836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:13:31.216701 21836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:35503
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:31.217917 21836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:31.219198 21836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:31.232657 21848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:31.232627 21846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:31.232867 21855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:31.233349 21836 server_base.cc:1034] running on GCE node
I20260811 00:13:31.233958 21836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:31.234758 21836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:31.235980 21836 hybrid_clock.cc:648] HybridClock initialized: now 1786407211235929 us; error 46 us; skew 500 ppm
I20260811 00:13:31.236265 21836 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:31 dist-test-slave-8hhm krb5kdc[21656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407211, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:31.239195 21836 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:13:31.240437 21836 webserver.cc:460] Webserver started at http://127.7.94.1:37825/ using document root <none> and password file <none>
I20260811 00:13:31.241097 21836 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:31.241273 21836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:31.241618 21836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:31.244357 21836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "8dcb3a1e894c462fad9d3fa4c84460a9"
format_stamp: "Formatted at 2026-08-11 00:13:31 on dist-test-slave-8hhm"
server_key: "919434fa0fd10a0f96a65001bfc97700"
server_key_iv: "712e2ccca07d310deb970f2f0c1983dc"
server_key_version: "encryptionkey@0"
I20260811 00:13:31.245126 21836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "8dcb3a1e894c462fad9d3fa4c84460a9"
format_stamp: "Formatted at 2026-08-11 00:13:31 on dist-test-slave-8hhm"
server_key: "919434fa0fd10a0f96a65001bfc97700"
server_key_iv: "712e2ccca07d310deb970f2f0c1983dc"
server_key_version: "encryptionkey@0"
I20260811 00:13:31.250034 21836 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 00:13:31.253459 21865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:31.254714 21836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:13:31.254884 21836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "8dcb3a1e894c462fad9d3fa4c84460a9"
format_stamp: "Formatted at 2026-08-11 00:13:31 on dist-test-slave-8hhm"
server_key: "919434fa0fd10a0f96a65001bfc97700"
server_key_iv: "712e2ccca07d310deb970f2f0c1983dc"
server_key_version: "encryptionkey@0"
I20260811 00:13:31.255089 21836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:31.283149 21836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:31.287617 21836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:31.287909 21836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:31.288837 21836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:31.290232 21836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:31.290355 21836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:31.290509 21836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:31.290591 21836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:31.324400 21836 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:46517
I20260811 00:13:31.324455 22055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:46517 every 8 connection(s)
I20260811 00:13:31.325790 21836 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:13:31.334434 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 21836
I20260811 00:13:31.334640 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:13:31.335382 7544 external_mini_cluster.cc:1428] Setting key bbbe1ed025fb2025bc8c7a2b95e35d2a
Aug 11 00:13:31 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407211, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:13:31.347481 22061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:31.328294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:37753 (local address 127.7.94.62:35503)
0811 00:13:31.328779 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:31.328783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:31.329620 (+ 837us) server_negotiation.cc:407] Connection header received
0811 00:13:31.331468 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:31.331474 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:31.331566 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:31.331708 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:31.333249 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:31.333946 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:31.335859 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:31.336360 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:31.336488 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:31.340444 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:31.340453 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:31.340461 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:31.340510 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:31.344220 (+ 3710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:31.344904 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:31.344913 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:31.344919 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:31.345065 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:31.345540 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:31.345547 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:31.345554 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:31.345953 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:31.346116 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:31.346763 (+ 647us) server_negotiation.cc:299] Negotiation successful
0811 00:13:31.347031 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":254,"threads_started":1}
I20260811 00:13:31.348099 22062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:31.328701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.1:37753)
0811 00:13:31.329447 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:31.329491 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:31.331198 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:31.331952 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:31.331961 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:31.332490 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:31.333047 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:31.333064 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:31.334136 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:31.334140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:31.335683 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:31.335696 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:31.336605 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:31.336613 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:31.336879 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:31.337722 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:31.337745 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:31.340190 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:31.344352 (+ 4162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:31.344359 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:31.344371 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:31.344779 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:31.345172 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:31.345176 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:31.345178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:31.345413 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:31.346093 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:31.346099 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:31.346388 (+ 289us) client_negotiation.cc:769] Sending connection context
0811 00:13:31.346607 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 00:13:31.346985 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":282,"threads_started":1}
I20260811 00:13:31.349685 22056 heartbeater.cc:347] Connected to a master server at 127.7.94.62:35503
I20260811 00:13:31.349993 22056 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:31.350870 22056 heartbeater.cc:511] Master 127.7.94.62:35503 requested a full tablet report, sending...
I20260811 00:13:31.353124 21726 ts_manager.cc:194] Registered new tserver with Master: 8dcb3a1e894c462fad9d3fa4c84460a9 (127.7.94.1:46517)
I20260811 00:13:31.354617 21726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:37753
WARNING: no policy specified for kudu/127.7.94.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
I20260811 00:13:31.394810 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:35503
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:31.732497 22068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:31.732767 22068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:31.732968 22068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:31.740073 22068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:31.740175 22068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:31.740319 22068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:13:31.749893 22068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:35503
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:31.751125 22068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:31.752439 22068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:31.765558 22081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:31.765558 22080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:31.765592 22083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:31.766492 22068 server_base.cc:1034] running on GCE node
I20260811 00:13:31.767163 22068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:31.768091 22068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:31.769330 22068 hybrid_clock.cc:648] HybridClock initialized: now 1786407211769267 us; error 55 us; skew 500 ppm
I20260811 00:13:31.769641 22068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:31 dist-test-slave-8hhm krb5kdc[21656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407211, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:31.772818 22068 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:13:31.774183 22068 webserver.cc:460] Webserver started at http://127.7.94.2:43967/ using document root <none> and password file <none>
I20260811 00:13:31.774850 22068 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:31.774991 22068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:31.775297 22068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:31.778205 22068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "ea169463fdb1428688945b3320734b22"
format_stamp: "Formatted at 2026-08-11 00:13:31 on dist-test-slave-8hhm"
server_key: "274ccbdecbecc2be58561fbe0f63c3ef"
server_key_iv: "c29a91ef3ba306f453786c0691f604b9"
server_key_version: "encryptionkey@0"
I20260811 00:13:31.778959 22068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "ea169463fdb1428688945b3320734b22"
format_stamp: "Formatted at 2026-08-11 00:13:31 on dist-test-slave-8hhm"
server_key: "274ccbdecbecc2be58561fbe0f63c3ef"
server_key_iv: "c29a91ef3ba306f453786c0691f604b9"
server_key_version: "encryptionkey@0"
I20260811 00:13:31.783916 22068 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:13:31.787243 22095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:31.788478 22068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:13:31.788691 22068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "ea169463fdb1428688945b3320734b22"
format_stamp: "Formatted at 2026-08-11 00:13:31 on dist-test-slave-8hhm"
server_key: "274ccbdecbecc2be58561fbe0f63c3ef"
server_key_iv: "c29a91ef3ba306f453786c0691f604b9"
server_key_version: "encryptionkey@0"
I20260811 00:13:31.788897 22068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:31.815934 22068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:31.819901 22068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:31.820189 22068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:31.821112 22068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:31.822414 22068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:31.822486 22068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:31.822678 22068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:31.822732 22068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:31.854964 22068 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:32895
I20260811 00:13:31.855051 22282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:32895 every 8 connection(s)
I20260811 00:13:31.856248 22068 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:13:31.859800 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 22068
I20260811 00:13:31.859994 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:13:31.860497 7544 external_mini_cluster.cc:1428] Setting key 0d66e1f4e1c6e894727c35942549e9c5
Aug 11 00:13:31 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407211, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:13:31.878309 22286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:31.858760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:45485 (local address 127.7.94.62:35503)
0811 00:13:31.859217 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:31.859222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:31.860345 (+ 1123us) server_negotiation.cc:407] Connection header received
0811 00:13:31.862427 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:31.862433 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:31.862504 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:31.862593 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:31.864769 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:31.865632 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:31.868021 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:31.868415 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:31.868501 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:31.872685 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:31.872690 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:31.872693 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:31.872723 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:31.875299 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:31.876006 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:31.876018 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:31.876021 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:31.876170 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:31.876601 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:31.876607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:31.876610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:31.876873 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:31.877023 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:31.877759 (+ 736us) server_negotiation.cc:299] Negotiation successful
0811 00:13:31.877955 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":221,"threads_started":1}
I20260811 00:13:31.879424 22287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:31.859342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.2:45485)
0811 00:13:31.860174 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:31.860219 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:31.862142 (+ 1923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:31.862874 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:31.862886 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:31.863700 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:31.864559 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:31.864589 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:31.865792 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:31.865799 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:31.867873 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:31.867891 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:31.868631 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:31.868640 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:31.868996 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:31.869750 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:31.869783 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:31.872467 (+ 2684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:31.875449 (+ 2982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:31.875457 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:31.875472 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:31.875883 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:31.876292 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:31.876296 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:31.876300 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:31.876485 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:31.877000 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:31.877007 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:31.877405 (+ 398us) client_negotiation.cc:769] Sending connection context
0811 00:13:31.877658 (+ 253us) client_negotiation.cc:240] Negotiation successful
0811 00:13:31.878140 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":371,"threads_started":1}
I20260811 00:13:31.880858 22284 heartbeater.cc:347] Connected to a master server at 127.7.94.62:35503
I20260811 00:13:31.881155 22284 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:31.881865 22284 heartbeater.cc:511] Master 127.7.94.62:35503 requested a full tablet report, sending...
I20260811 00:13:31.883312 21726 ts_manager.cc:194] Registered new tserver with Master: ea169463fdb1428688945b3320734b22 (127.7.94.2:32895)
I20260811 00:13:31.884114 21726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:45485
WARNING: no policy specified for kudu/127.7.94.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
I20260811 00:13:31.920591 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:35503
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:32.252457 22291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:32.252751 22291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:32.252848 22291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:32.260033 22291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:32.260146 22291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:32.260264 22291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:13:32.270148 22291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:35503
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:32.271478 22291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:32.272764 22291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:32.286186 22306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:32.286271 22301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:32.286247 22308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:32.287230 22291 server_base.cc:1034] running on GCE node
I20260811 00:13:32.287883 22291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:32.288753 22291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:32.289983 22291 hybrid_clock.cc:648] HybridClock initialized: now 1786407212289939 us; error 52 us; skew 500 ppm
I20260811 00:13:32.290297 22291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:32 dist-test-slave-8hhm krb5kdc[21656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407212, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:32.293735 22291 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:13:32.295060 22291 webserver.cc:460] Webserver started at http://127.7.94.3:37233/ using document root <none> and password file <none>
I20260811 00:13:32.295703 22291 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:32.295804 22291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:32.296061 22291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:32.298965 22291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "7856d3726a524c26a669f54340b5a570"
format_stamp: "Formatted at 2026-08-11 00:13:32 on dist-test-slave-8hhm"
server_key: "d4cd1be4a728217c2049c0574f1252a3"
server_key_iv: "089a7e45598c9125fd792907b8a21fe6"
server_key_version: "encryptionkey@0"
I20260811 00:13:32.299713 22291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "7856d3726a524c26a669f54340b5a570"
format_stamp: "Formatted at 2026-08-11 00:13:32 on dist-test-slave-8hhm"
server_key: "d4cd1be4a728217c2049c0574f1252a3"
server_key_iv: "089a7e45598c9125fd792907b8a21fe6"
server_key_version: "encryptionkey@0"
I20260811 00:13:32.305305 22291 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260811 00:13:32.308692 22318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:32.310015 22291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 00:13:32.310200 22291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "7856d3726a524c26a669f54340b5a570"
format_stamp: "Formatted at 2026-08-11 00:13:32 on dist-test-slave-8hhm"
server_key: "d4cd1be4a728217c2049c0574f1252a3"
server_key_iv: "089a7e45598c9125fd792907b8a21fe6"
server_key_version: "encryptionkey@0"
I20260811 00:13:32.310401 22291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:32.332274 22291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:32.336205 22291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:32.336493 22291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:32.337517 22291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:32.338826 22291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:32.338901 22291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:32.339072 22291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:32.339123 22291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:32.358254 22056 heartbeater.cc:503] Master 127.7.94.62:35503 was elected leader, sending a full tablet report...
I20260811 00:13:32.372905 22291 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:34261
I20260811 00:13:32.372987 22502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:34261 every 8 connection(s)
I20260811 00:13:32.374234 22291 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:13:32.375818 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 22291
I20260811 00:13:32.376000 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:13:32.376590 7544 external_mini_cluster.cc:1428] Setting key fee731ce8d020b560a63ea7d65387889
Aug 11 00:13:32 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407212, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:13:32.394697 22286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:32.377360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:57461 (local address 127.7.94.62:35503)
0811 00:13:32.377561 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:32.377567 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:32.378965 (+ 1398us) server_negotiation.cc:407] Connection header received
0811 00:13:32.381088 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:32.381092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:32.381205 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:32.381347 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:32.382799 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:32.383475 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:32.384983 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:32.385452 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:32.385545 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:32.388909 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:32.388914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:32.388917 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:32.388946 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:32.391792 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:32.392684 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:32.392688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:32.392690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:32.392813 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:32.393174 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:32.393177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:32.393179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:32.393508 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:32.393638 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:32.394288 (+ 650us) server_negotiation.cc:299] Negotiation successful
0811 00:13:32.394463 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 00:13:32.395637 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:32.377948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.3:57461)
0811 00:13:32.378813 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:32.378852 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:32.380814 (+ 1962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:32.381545 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:32.381553 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:32.382092 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:32.382656 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:32.382673 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:32.383603 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:32.383606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:32.384861 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:32.384872 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:32.385636 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:32.385642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:32.385827 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:32.386511 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:32.386533 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:32.388713 (+ 2180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:32.391943 (+ 3230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:32.391950 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:32.391962 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:32.392573 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:32.392912 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:32.392915 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:32.392918 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:32.393090 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:32.393654 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:32.393668 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:32.393965 (+ 297us) client_negotiation.cc:769] Sending connection context
0811 00:13:32.394202 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 00:13:32.394569 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":343,"threads_started":1}
I20260811 00:13:32.397037 22503 heartbeater.cc:347] Connected to a master server at 127.7.94.62:35503
I20260811 00:13:32.397394 22503 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:32.398094 22503 heartbeater.cc:511] Master 127.7.94.62:35503 requested a full tablet report, sending...
I20260811 00:13:32.399504 21726 ts_manager.cc:194] Registered new tserver with Master: 7856d3726a524c26a669f54340b5a570 (127.7.94.3:34261)
I20260811 00:13:32.400285 21726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:57461
I20260811 00:13:32.404337 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.1.keytab.
Entry for principal oryx/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.1.keytab.
WARNING: no policy specified for oryx/127.7.94.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.94.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.2.keytab.
Entry for principal oryx/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.2.keytab.
WARNING: no policy specified for oryx/127.7.94.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.94.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.3.keytab.
Entry for principal oryx/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.3.keytab.
I20260811 00:13:32.562053 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 21836
Aug 11 00:13:32 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:13:32.578945 22524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:32.563343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46000 (local address 127.7.94.1:46517)
0811 00:13:32.564163 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:32.564171 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:32.564199 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:13:32.564262 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:32.564269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:32.564504 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:32.564640 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:32.566079 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:32.567081 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:32.568234 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:32.568778 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:32.568905 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:32.573097 (+ 4192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:32.573108 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:32.573123 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:32.573160 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:32.576329 (+ 3169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:32.576966 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:32.576978 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:32.576980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:32.577138 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:32.577555 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:32.577560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:32.577563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:32.577931 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:32.578169 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:32.578387 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 00:13:32.578589 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":452,"threads_started":1}
I20260811 00:13:32.580255 22046 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:32.663621 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 21836
I20260811 00:13:32.689740 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 22068
Aug 11 00:13:32 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.2@KRBTEST.COM
I20260811 00:13:32.705399 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:32.690665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36248 (local address 127.7.94.2:32895)
0811 00:13:32.691200 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:32.691207 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:32.691234 (+ 27us) server_negotiation.cc:407] Connection header received
0811 00:13:32.691296 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:32.691302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:32.691504 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:32.691647 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:32.693019 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:32.694087 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:32.695419 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:32.695950 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:32.696044 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:32.700089 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:32.700107 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:32.700120 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:32.700157 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:32.702966 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:32.703607 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:32.703614 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:32.703616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:32.703738 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:32.704039 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:32.704042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:32.704044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:32.704379 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:32.704610 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:32.704836 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 00:13:32.705025 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":271,"threads_started":1}
I20260811 00:13:32.706624 22275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:32.790524 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 22068
I20260811 00:13:32.817392 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 22291
Aug 11 00:13:32 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.3@KRBTEST.COM
I20260811 00:13:32.833921 22531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:32.818436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49638 (local address 127.7.94.3:34261)
0811 00:13:32.818948 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:32.818955 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:32.818985 (+ 30us) server_negotiation.cc:407] Connection header received
0811 00:13:32.819046 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:32.819052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:32.819257 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:32.819422 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:32.820762 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:32.822007 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:32.823350 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:32.823903 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:32.824042 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:32.828277 (+ 4235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:32.828288 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:32.828302 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:32.828336 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:32.831158 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:32.831836 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:32.831843 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:32.831845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:32.831968 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:32.832341 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:32.832345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:32.832349 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:32.832688 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:32.832996 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:32.833306 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 00:13:32.833576 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":232,"threads_started":1}
I20260811 00:13:32.835240 22496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:32.922381 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 22291
I20260811 00:13:32.947898 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 21670
I20260811 00:13:32.949131 21779 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:33.053553 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 21670
I20260811 00:13:33.087081 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:35503
--webserver_interface=127.7.94.62
--webserver_port=34241
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:35503
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:33.414610 22536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:33.414968 22536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:33.415129 22536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:33.422231 22536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:13:33.422371 22536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:33.422477 22536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:33.422582 22536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:13:33.422675 22536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:13:33.432199 22536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:35503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:35503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=34241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:33.433521 22536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:33.434813 22536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:33.446523 22547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:33.446580 22543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:33.446555 22545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:33.447458 22536 server_base.cc:1034] running on GCE node
I20260811 00:13:33.448239 22536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:33.449648 22536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:33.451035 22536 hybrid_clock.cc:648] HybridClock initialized: now 1786407213450901 us; error 117 us; skew 500 ppm
I20260811 00:13:33.451392 22536 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:33 dist-test-slave-8hhm krb5kdc[21656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407213, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:33.454631 22536 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:13:33.456188 22536 webserver.cc:460] Webserver started at http://127.7.94.62:34241/ using document root <none> and password file <none>
I20260811 00:13:33.456862 22536 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:33.457026 22536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:33.462464 22536 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:13:33.465462 22562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:33.466827 22536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:13:33.467021 22536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "890248e3001045859606028605dde43e"
format_stamp: "Formatted at 2026-08-11 00:13:30 on dist-test-slave-8hhm"
server_key: "19f612e920fbe78ce3d401da2e812f79"
server_key_iv: "7a957f253e6ec0d3507b40047c51d0c5"
server_key_version: "encryptionkey@0"
I20260811 00:13:33.468010 22536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:33.480310 22536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:33.484591 22536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:33.484864 22536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:33.503052 22536 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:35503
I20260811 00:13:33.503098 22650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:35503 every 8 connection(s)
I20260811 00:13:33.504540 22536 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:13:33.511052 22652 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:13:33.512735 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 22536
I20260811 00:13:33.513746 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:46517
--local_ip_for_outbound_sockets=127.7.94.1
--tserver_master_addrs=127.7.94.62:35503
--webserver_port=37825
--webserver_interface=127.7.94.1
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.94.1
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.1.keytab with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:13:33.515304 22652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e: Bootstrap starting.
I20260811 00:13:33.536990 22652 log.cc:824] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e: Log is configured to *not* fsync() on all Append() calls
I20260811 00:13:33.545640 22652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:13:33.546196 22652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e: Bootstrap complete.
I20260811 00:13:33.555874 22652 raft_consensus.cc:348] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890248e3001045859606028605dde43e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35503 } }
I20260811 00:13:33.556649 22652 raft_consensus.cc:729] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 890248e3001045859606028605dde43e, State: Initialized, Role: FOLLOWER
I20260811 00:13:33.557364 22652 consensus_queue.cc:260] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890248e3001045859606028605dde43e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35503 } }
I20260811 00:13:33.557597 22652 raft_consensus.cc:388] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:13:33.557709 22652 raft_consensus.cc:482] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:13:33.557904 22652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [term 1 FOLLOWER]: Advancing to term 2
I20260811 00:13:33.562098 22652 raft_consensus.cc:504] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890248e3001045859606028605dde43e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35503 } }
I20260811 00:13:33.562592 22652 leader_election.cc:302] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 890248e3001045859606028605dde43e; no voters:
I20260811 00:13:33.563213 22652 leader_election.cc:288] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 00:13:33.563336 22657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [term 2 FOLLOWER]: Leader election won for term 2
I20260811 00:13:33.565598 22652 sys_catalog.cc:564] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:13:33.570437 22657 raft_consensus.cc:686] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [term 2 LEADER]: Becoming Leader. State: Replica: 890248e3001045859606028605dde43e, State: Running, Role: LEADER
I20260811 00:13:33.571108 22657 consensus_queue.cc:237] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890248e3001045859606028605dde43e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35503 } }
I20260811 00:13:33.574009 22658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "890248e3001045859606028605dde43e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890248e3001045859606028605dde43e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35503 } } }
I20260811 00:13:33.574496 22658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:33.575453 22675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:13:33.575228 22661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 890248e3001045859606028605dde43e. Latest consensus state: current_term: 2 leader_uuid: "890248e3001045859606028605dde43e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890248e3001045859606028605dde43e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35503 } } }
I20260811 00:13:33.576345 22661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:33.590620 22675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:13:33.592136 22675 catalog_manager.cc:1259] Loaded cluster ID: a3b6b7c5d24d4b04b051575aa7fccbd6
I20260811 00:13:33.592303 22675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:13:33.596403 22675 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:13:33.597956 22675 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 890248e3001045859606028605dde43e: Loaded TSK: 0
I20260811 00:13:33.599586 22675 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 00:13:33.869431 22654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:33.869758 22654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:33.869926 22654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:33.877417 22654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:33.877573 22654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:33.877744 22654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:13:33.887813 22654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.1.keytab
--principal=oryx/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:46517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=37825
--tserver_master_addrs=127.7.94.62:35503
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:33.889183 22654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:33.890580 22654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:33.904609 22697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:33.905093 22696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:33.905009 22703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:33.905901 22654 server_base.cc:1034] running on GCE node
I20260811 00:13:33.906548 22654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:33.907482 22654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:33.908716 22654 hybrid_clock.cc:648] HybridClock initialized: now 1786407213908660 us; error 49 us; skew 500 ppm
I20260811 00:13:33.909066 22654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:33 dist-test-slave-8hhm krb5kdc[21656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407213, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:33.917922 22654 init.cc:377] Logged in from keytab as oryx/127.7.94.1@KRBTEST.COM (short username oryx)
I20260811 00:13:33.919649 22654 webserver.cc:460] Webserver started at http://127.7.94.1:37825/ using document root <none> and password file <none>
I20260811 00:13:33.920372 22654 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:33.920631 22654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:33.926434 22654 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 00:13:33.929364 22718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:33.931031 22654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:13:33.931226 22654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "8dcb3a1e894c462fad9d3fa4c84460a9"
format_stamp: "Formatted at 2026-08-11 00:13:31 on dist-test-slave-8hhm"
server_key: "919434fa0fd10a0f96a65001bfc97700"
server_key_iv: "712e2ccca07d310deb970f2f0c1983dc"
server_key_version: "encryptionkey@0"
I20260811 00:13:33.932248 22654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:33.951349 22654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:33.955294 22654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:33.955566 22654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:33.956522 22654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:33.957909 22654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:33.958040 22654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:33.958182 22654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:33.958279 22654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:33.992301 22654 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:46517
I20260811 00:13:33.992391 22910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:46517 every 8 connection(s)
I20260811 00:13:33.993783 22654 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:13:34.003937 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 22654
I20260811 00:13:34.004709 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:32895
--local_ip_for_outbound_sockets=127.7.94.2
--tserver_master_addrs=127.7.94.62:35503
--webserver_port=43967
--webserver_interface=127.7.94.2
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.94.2
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.2.keytab with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:13:34 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:34 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:34.014218 22915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:33.996635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.1:54021)
0811 00:13:33.997477 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:33.997514 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:33.999105 (+ 1591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:34.000601 (+ 1496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:34.000613 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:34.001413 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:34.002179 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:34.002207 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.004240 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.004247 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:34.006405 (+ 2158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:34.006425 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.007478 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.007489 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:34.007822 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:34.008544 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:34.008584 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:34.012824 (+ 4240us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":381,"threads_started":1}
W20260811 00:13:34.014310 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:33.996602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:54021 (local address 127.7.94.62:35503)
0811 00:13:33.997368 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:33.997382 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:33.997635 (+ 253us) server_negotiation.cc:407] Connection header received
0811 00:13:33.999576 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:33.999588 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:33.999902 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:34.000367 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:34.002435 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.004024 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.006553 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.007213 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.007342 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:34.013299 (+ 5957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.1:54021: BlockingRecv error: recv got EOF from 127.7.94.1:54021 (error 108)
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":248,"threads_started":1}
W20260811 00:13:34.015192 22911 heartbeater.cc:644] Failed to heartbeat to 127.7.94.62:35503 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.94.62:35503: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Aug 11 00:13:34 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:34 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:34.024029 22915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:34.015845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.1:45525)
0811 00:13:34.016003 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:34.016021 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:34.016128 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:34.016486 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:34.016489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:34.016788 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:34.016921 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:34.016928 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.018461 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.018465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:34.019349 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:34.019361 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.020378 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.020382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:34.020496 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:34.021087 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:34.021115 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:34.023837 (+ 2722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 00:13:34.024370 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:34.016003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:45525 (local address 127.7.94.62:35503)
0811 00:13:34.016187 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:34.016194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:34.016214 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:13:34.016273 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:34.016279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:34.016355 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:34.016467 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:34.017063 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.018323 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.019498 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.020242 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.020379 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:34.024103 (+ 3724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.1:45525: BlockingRecv error: recv got EOF from 127.7.94.1:45525 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 00:13:34 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:34 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 00:13:34.034520 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:34.025032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:60673 (local address 127.7.94.62:35503)
0811 00:13:34.025185 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:34.025189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:34.025204 (+ 15us) server_negotiation.cc:407] Connection header received
0811 00:13:34.025331 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:34.025334 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:34.025400 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:34.025475 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:34.026314 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.026993 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.029059 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.029510 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.029635 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:34.034334 (+ 4699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.1:60673: BlockingRecv error: recv got EOF from 127.7.94.1:60673 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 00:13:34.034525 22915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:34.024844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.1:60673)
0811 00:13:34.025072 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:34.025098 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:34.025228 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:34.025531 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:34.025539 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:34.025959 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:34.026172 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:34.026187 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.027144 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.027150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:34.028917 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:34.028937 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.029688 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.029698 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:34.029923 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:34.030526 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:34.030556 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:34.034178 (+ 3622us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 00:13:34.034878 22911 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 00:13:34.346118 22916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:34.346451 22916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:34.346616 22916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:34.353780 22916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:34.353919 22916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:34.354100 22916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:13:34.368826 22916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.2.keytab
--principal=oryx/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:32895
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=43967
--tserver_master_addrs=127.7.94.62:35503
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:34.370558 22916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:34.372249 22916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:34.389189 22935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:34.389339 22927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:34.389359 22928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:34.390091 22916 server_base.cc:1034] running on GCE node
I20260811 00:13:34.390746 22916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:34.391752 22916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:34.392998 22916 hybrid_clock.cc:648] HybridClock initialized: now 1786407214392938 us; error 56 us; skew 500 ppm
I20260811 00:13:34.393384 22916 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:34 dist-test-slave-8hhm krb5kdc[21656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407214, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:34.396727 22916 init.cc:377] Logged in from keytab as oryx/127.7.94.2@KRBTEST.COM (short username oryx)
I20260811 00:13:34.400874 22916 webserver.cc:460] Webserver started at http://127.7.94.2:43967/ using document root <none> and password file <none>
I20260811 00:13:34.401846 22916 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:34.402073 22916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:34.409431 22916 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:13:34.413148 22944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:34.414983 22916 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:13:34.415186 22916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "ea169463fdb1428688945b3320734b22"
format_stamp: "Formatted at 2026-08-11 00:13:31 on dist-test-slave-8hhm"
server_key: "274ccbdecbecc2be58561fbe0f63c3ef"
server_key_iv: "c29a91ef3ba306f453786c0691f604b9"
server_key_version: "encryptionkey@0"
I20260811 00:13:34.416637 22916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:34.447582 22916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:34.452116 22916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:34.452410 22916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:34.453451 22916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:34.454747 22916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:34.454871 22916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:34.454991 22916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:34.455077 22916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:34.489149 22916 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:32895
I20260811 00:13:34.489172 23154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:32895 every 8 connection(s)
I20260811 00:13:34.490706 22916 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:13:34.496563 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 22916
I20260811 00:13:34.497184 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:34261
--local_ip_for_outbound_sockets=127.7.94.3
--tserver_master_addrs=127.7.94.62:35503
--webserver_port=37233
--webserver_interface=127.7.94.3
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.94.3
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.3.keytab with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:13:34 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:34 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 00:13:34.509060 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:34.493333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:45737 (local address 127.7.94.62:35503)
0811 00:13:34.493557 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:34.493564 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:34.494873 (+ 1309us) server_negotiation.cc:407] Connection header received
0811 00:13:34.497185 (+ 2312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:34.497191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:34.497323 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:34.497446 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:34.499116 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.500204 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.501854 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.502553 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.502708 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:34.508848 (+ 6140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.2:45737: BlockingRecv error: recv got EOF from 127.7.94.2:45737 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 00:13:34.509822 23161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:34.493947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.2:45737)
0811 00:13:34.494702 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:34.494740 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:34.496913 (+ 2173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:34.497771 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:34.497779 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:34.498319 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:34.498867 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:34.498884 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.500375 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.500379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:34.501695 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:34.501708 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.502824 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.502832 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:34.503217 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:34.504066 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:34.504097 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:34.508594 (+ 4497us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":286,"threads_started":1}
W20260811 00:13:34.510504 23156 heartbeater.cc:644] Failed to heartbeat to 127.7.94.62:35503 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.94.62:35503: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Aug 11 00:13:34 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:34 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:34.521147 23161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:34.511223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.2:50581)
0811 00:13:34.511481 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:34.511502 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:34.511614 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:34.511959 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:34.511967 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:34.512448 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:34.512648 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:34.512672 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.514129 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.514137 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:34.515806 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:34.515824 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.516520 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.516528 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:34.516769 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:34.517471 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:34.517496 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:34.520882 (+ 3386us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134}
W20260811 00:13:34.521677 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:34.511283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:50581 (local address 127.7.94.62:35503)
0811 00:13:34.511431 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:34.511436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:34.511585 (+ 149us) server_negotiation.cc:407] Connection header received
0811 00:13:34.511741 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:34.511746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:34.511816 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:34.511911 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:34.512826 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.513969 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.515958 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.516334 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.516455 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:34.521164 (+ 4709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.2:50581: BlockingRecv error: recv got EOF from 127.7.94.2:50581 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 00:13:34 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:34 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:34.531385 23161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:34.521910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.2:53389)
0811 00:13:34.522117 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:34.522138 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:34.522250 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:34.522718 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:34.522724 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:34.523153 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:34.523402 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:34.523417 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.524390 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.524398 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:34.526196 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:34.526227 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.526851 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.526859 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:34.527069 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:34.527656 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:34.527693 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:34.531084 (+ 3391us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 00:13:34.531631 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:34.522298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:53389 (local address 127.7.94.62:35503)
0811 00:13:34.522452 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:34.522456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:34.522470 (+ 14us) server_negotiation.cc:407] Connection header received
0811 00:13:34.522510 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:34.522513 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:34.522565 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:34.522641 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:34.523556 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.524245 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.526344 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.526707 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.526794 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:34.531287 (+ 4493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.2:53389: BlockingRecv error: recv got EOF from 127.7.94.2:53389 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 00:13:34.531724 23156 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 00:13:34.843098 23162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:34.843426 23162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:34.843605 23162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:34.850914 23162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:34.851107 23162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:34.851290 23162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:13:34.861110 23162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/krb5kdc/oryx_127.7.94.3.keytab
--principal=oryx/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:34261
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=37233
--tserver_master_addrs=127.7.94.62:35503
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:34.862529 23162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:34.863937 23162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:34.877733 23172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:34.877745 23177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:34.877799 23173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:34.878577 23162 server_base.cc:1034] running on GCE node
I20260811 00:13:34.879123 23162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:34.880018 23162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:34.881258 23162 hybrid_clock.cc:648] HybridClock initialized: now 1786407214881160 us; error 57 us; skew 500 ppm
I20260811 00:13:34.881692 23162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:34 dist-test-slave-8hhm krb5kdc[21656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407214, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:34.885944 23162 init.cc:377] Logged in from keytab as oryx/127.7.94.3@KRBTEST.COM (short username oryx)
I20260811 00:13:34.887813 23162 webserver.cc:460] Webserver started at http://127.7.94.3:37233/ using document root <none> and password file <none>
I20260811 00:13:34.888778 23162 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:34.888963 23162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:34.895107 23162 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 00:13:34.897857 23191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:34.899217 23162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:13:34.899390 23162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "7856d3726a524c26a669f54340b5a570"
format_stamp: "Formatted at 2026-08-11 00:13:32 on dist-test-slave-8hhm"
server_key: "d4cd1be4a728217c2049c0574f1252a3"
server_key_iv: "089a7e45598c9125fd792907b8a21fe6"
server_key_version: "encryptionkey@0"
I20260811 00:13:34.900540 23162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:34.913753 23162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:34.917591 23162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:34.917873 23162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:34.918835 23162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:34.920305 23162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:34.920420 23162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:34.920535 23162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:34.920648 23162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:34.953862 23162 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:34261
I20260811 00:13:34.953948 23401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:34261 every 8 connection(s)
I20260811 00:13:34.955415 23162 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:13:34.960449 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 23162
Aug 11 00:13:34 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:34 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 00:13:34.973742 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:34.957994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:42355 (local address 127.7.94.62:35503)
0811 00:13:34.958178 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:34.958182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:34.959390 (+ 1208us) server_negotiation.cc:407] Connection header received
0811 00:13:34.961443 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:34.961450 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:34.961548 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:34.961668 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:34.964338 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.965087 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.966775 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.967550 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.967695 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:34.973542 (+ 5847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.3:42355: BlockingRecv error: recv got EOF from 127.7.94.3:42355 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 00:13:34.974622 23408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:34.958490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.3:42355)
0811 00:13:34.959222 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:34.959260 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:34.961048 (+ 1788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:34.962194 (+ 1146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:34.962210 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:34.963168 (+ 958us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:34.964120 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:34.964150 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.965292 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.965300 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:34.966595 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:34.966607 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.967805 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.967814 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:34.968135 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:34.968894 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:34.968930 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:34.973411 (+ 4481us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":282,"threads_started":1}
W20260811 00:13:34.975374 23402 heartbeater.cc:644] Failed to heartbeat to 127.7.94.62:35503 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.94.62:35503: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
I20260811 00:13:34.981387 23411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:34.961543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59370 (local address 127.7.94.62:35503)
0811 00:13:34.965528 (+ 3985us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:34.965532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:34.965546 (+ 14us) server_negotiation.cc:407] Connection header received
0811 00:13:34.965585 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:34.965588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:34.965644 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:34.965771 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:34.967730 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.968815 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.970157 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.970666 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.970800 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:34.974205 (+ 3405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:34.974213 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:34.974230 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:34.974270 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:34.977975 (+ 3705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:34.978690 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:34.978699 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:34.978707 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:34.978901 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:34.979292 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:34.979297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:34.979300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:34.979775 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:34.980081 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:34.980552 (+ 471us) server_negotiation.cc:299] Negotiation successful
0811 00:13:34.980941 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3898,"thread_start_us":3813,"threads_started":1}
Aug 11 00:13:34 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:34 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:34.984124 23408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:34.975999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.3:45505)
0811 00:13:34.976178 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:34.976201 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:34.976303 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:34.976616 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:34.976622 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:34.977021 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:34.977231 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:34.977244 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.978229 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.978234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:34.979110 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:34.979120 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.979822 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.979828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:34.980003 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:34.980495 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:34.980516 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:34.983843 (+ 3327us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 00:13:34.984284 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:34.976076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:45505 (local address 127.7.94.62:35503)
0811 00:13:34.976255 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:34.976259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:34.976274 (+ 15us) server_negotiation.cc:407] Connection header received
0811 00:13:34.976413 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:34.976417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:34.976469 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:34.976543 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:34.977381 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.978054 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.979249 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.979675 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.979809 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:34.984036 (+ 4227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.3:45505: BlockingRecv error: recv got EOF from 127.7.94.3:45505 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 00:13:34 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:34 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:34.992408 23408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:34.984949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.3:38987)
0811 00:13:34.985093 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:34.985115 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:34.985247 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:34.985573 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:34.985578 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:34.985880 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:34.986048 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:34.986056 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.987006 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.987012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:34.987894 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:34.987905 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.988563 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.988569 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:34.988734 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:34.989317 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:34.989333 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:34.992154 (+ 2821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 00:13:34.992600 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:34.985023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:38987 (local address 127.7.94.62:35503)
0811 00:13:34.985288 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:34.985295 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:34.985312 (+ 17us) server_negotiation.cc:407] Connection header received
0811 00:13:34.985360 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:34.985364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:34.985448 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:34.985547 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:34.986182 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.986866 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.988029 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:34.988426 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:34.988550 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:34.992299 (+ 3749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.3:38987: BlockingRecv error: recv got EOF from 127.7.94.3:38987 (error 108)
Metrics: {"server-negotiator.queue_time_us":140}
W20260811 00:13:34.992811 23402 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 00:13:35 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:35 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:35.043618 23412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:35.035410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.1:52711)
0811 00:13:35.035931 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:35.035947 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:35.036133 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:35.036438 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:35.036441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:35.036946 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:35.037204 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:35.037230 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:35.038287 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:35.038290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:35.039252 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:35.039264 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:35.039947 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:35.039951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:35.040070 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:35.040746 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:35.040764 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:35.043411 (+ 2647us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":275,"threads_started":1}
W20260811 00:13:35.043982 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:35.035460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:52711 (local address 127.7.94.62:35503)
0811 00:13:35.035673 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:35.035679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:35.036049 (+ 370us) server_negotiation.cc:407] Connection header received
0811 00:13:35.036249 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:35.036253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:35.036315 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:35.036396 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:35.037353 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:35.038129 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:35.039376 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:35.039798 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:35.039888 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:35.043608 (+ 3720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.1:52711: BlockingRecv error: recv got EOF from 127.7.94.1:52711 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 00:13:35 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:35 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:35.540315 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:35.532285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.2:33433)
0811 00:13:35.532846 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:35.532862 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:35.532982 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:35.533302 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:35.533305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:35.533805 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:35.533991 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:35.533999 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:35.534925 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:35.534928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:35.535914 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:35.535925 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:35.536664 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:35.536668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:35.536796 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:35.537486 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:35.537502 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:35.540093 (+ 2591us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":306,"threads_started":1}
W20260811 00:13:35.540524 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:35.532351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:33433 (local address 127.7.94.62:35503)
0811 00:13:35.532524 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:35.532528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:35.532942 (+ 414us) server_negotiation.cc:407] Connection header received
0811 00:13:35.533076 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:35.533079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:35.533144 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:35.533280 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:35.534119 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:35.534799 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:35.536050 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:35.536519 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:35.536608 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:35.540296 (+ 3688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.2:33433: BlockingRecv error: recv got EOF from 127.7.94.2:33433 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 00:13:35 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:36 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:36.002177 23416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:35.993555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.3:58805)
0811 00:13:35.994273 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:35.994289 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:35.994400 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:35.994697 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:35.994700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:35.995165 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:35.995373 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:35.995382 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:35.996371 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:35.996375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:35.997414 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:35.997426 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:35.998065 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:35.998069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:35.998192 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:35.998810 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:35.998827 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:36.001951 (+ 3124us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":477,"threads_started":1}
W20260811 00:13:36.002377 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:35.993616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:58805 (local address 127.7.94.62:35503)
0811 00:13:35.993858 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:35.993867 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:35.994374 (+ 507us) server_negotiation.cc:407] Connection header received
0811 00:13:35.994502 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:35.994506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:35.994567 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:35.994650 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:35.995496 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:35.996244 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:35.997533 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:35.997906 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:35.998013 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:36.002159 (+ 4146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.3:58805: BlockingRecv error: recv got EOF from 127.7.94.3:58805 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 00:13:36 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:36 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:36.053455 23417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:36.044786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.1:54219)
0811 00:13:36.045405 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:36.045422 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:36.045572 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:36.045898 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:36.045901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:36.046435 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:36.046605 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:36.046614 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:36.047839 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:36.047844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:36.048819 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:36.048831 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:36.049530 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:36.049535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:36.049655 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:36.050235 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:36.050252 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:36.053137 (+ 2885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":347,"threads_started":1}
W20260811 00:13:36.053609 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:36.044917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:54219 (local address 127.7.94.62:35503)
0811 00:13:36.045116 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:36.045121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:36.045544 (+ 423us) server_negotiation.cc:407] Connection header received
0811 00:13:36.045686 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:36.045690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:36.045754 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:36.045837 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:36.046758 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:36.047691 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:36.048940 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:36.049397 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:36.049490 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:36.053371 (+ 3881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.1:54219: BlockingRecv error: recv got EOF from 127.7.94.1:54219 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 00:13:36 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:36 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:36.549705 23418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:36.541286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.2:49845)
0811 00:13:36.541807 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:36.541823 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:36.541981 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:36.542354 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:36.542376 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:36.542828 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:36.542974 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:36.542982 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:36.543995 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:36.544001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:36.544871 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:36.544882 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:36.545655 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:36.545661 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:36.545833 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:36.546460 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:36.546477 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:36.549358 (+ 2881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":302,"threads_started":1}
W20260811 00:13:36.549907 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:36.541337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:49845 (local address 127.7.94.62:35503)
0811 00:13:36.541552 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:36.541561 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:36.541912 (+ 351us) server_negotiation.cc:407] Connection header received
0811 00:13:36.542106 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:36.542112 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:36.542199 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:36.542316 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:36.543120 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:36.543835 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:36.545008 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:36.545500 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:36.545635 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:36.549595 (+ 3960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.2:49845: BlockingRecv error: recv got EOF from 127.7.94.2:49845 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 00:13:37 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:37 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:37.011730 23420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:37.003011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.3:59755)
0811 00:13:37.003566 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:37.003583 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:37.003737 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:37.004079 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:37.004084 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:37.004586 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:37.004733 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:37.004741 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:37.005842 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:37.005848 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:37.006730 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:37.006741 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:37.007435 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:37.007441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:37.007600 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:37.008267 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:37.008285 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:37.011355 (+ 3070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":307,"threads_started":1}
W20260811 00:13:37.011834 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:37.003183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:59755 (local address 127.7.94.62:35503)
0811 00:13:37.003402 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:37.003408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:37.003678 (+ 270us) server_negotiation.cc:407] Connection header received
0811 00:13:37.003857 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:37.003862 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:37.003927 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:37.004032 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:37.004882 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:37.005697 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:37.006869 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:37.007272 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:37.007406 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:37.011568 (+ 4162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.3:59755: BlockingRecv error: recv got EOF from 127.7.94.3:59755 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 00:13:37 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:37 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:37.062886 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:37.054357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.1:44853)
0811 00:13:37.054872 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:37.054888 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:37.055011 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:37.055284 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:37.055287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:37.055633 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:37.055801 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:37.055810 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:37.056847 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:37.056852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:37.057815 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:37.057828 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:37.058543 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:37.058549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:37.058730 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:37.059327 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:37.059345 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:37.062524 (+ 3179us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":263,"threads_started":1}
W20260811 00:13:37.063061 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:37.054445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:44853 (local address 127.7.94.62:35503)
0811 00:13:37.054766 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:37.054771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:37.054940 (+ 169us) server_negotiation.cc:407] Connection header received
0811 00:13:37.055094 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:37.055097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:37.055155 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:37.055233 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:37.055959 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:37.056678 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:37.057974 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:37.058386 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:37.058523 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:37.062790 (+ 4267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.1:44853: BlockingRecv error: recv got EOF from 127.7.94.1:44853 (error 108)
Metrics: {"server-negotiator.queue_time_us":192}
Aug 11 00:13:37 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:37 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:37.559410 23423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:37.550953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.2:49527)
0811 00:13:37.551524 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:37.551541 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:37.551665 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:37.551967 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:37.551971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:37.552308 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:37.552509 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:37.552521 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:37.554004 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:37.554008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:37.554884 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:37.554895 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:37.555519 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:37.555523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:37.555643 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:37.556279 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:37.556296 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:37.559175 (+ 2879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":299,"threads_started":1}
W20260811 00:13:37.559592 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:37.551015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:49527 (local address 127.7.94.62:35503)
0811 00:13:37.551189 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:37.551193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:37.551624 (+ 431us) server_negotiation.cc:407] Connection header received
0811 00:13:37.551778 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:37.551782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:37.551843 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:37.551923 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:37.552688 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:37.553865 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:37.555001 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:37.555388 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:37.555491 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:37.559371 (+ 3880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.2:49527: BlockingRecv error: recv got EOF from 127.7.94.2:49527 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 00:13:38 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:38 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:38.022608 23424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:38.012833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.3:60329)
0811 00:13:38.013381 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:38.013397 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:38.013572 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:38.013919 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:38.013923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:38.014313 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:38.014540 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:38.014556 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:38.016121 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:38.016127 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:38.018000 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:38.018014 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:38.018661 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:38.018665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:38.018785 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:38.019439 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:38.019458 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:38.022351 (+ 2893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":282,"threads_started":1}
W20260811 00:13:38.022783 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:38.012919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:60329 (local address 127.7.94.62:35503)
0811 00:13:38.013164 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:38.013172 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:38.013493 (+ 321us) server_negotiation.cc:407] Connection header received
0811 00:13:38.013710 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:38.013715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:38.013784 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:38.013885 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:38.014745 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:38.015971 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:38.018133 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:38.018527 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:38.018625 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:38.022581 (+ 3956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.3:60329: BlockingRecv error: recv got EOF from 127.7.94.3:60329 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 11 00:13:38 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:38 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:38.072043 23425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:38.064000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.1:57257)
0811 00:13:38.064610 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:38.064627 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:38.064741 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:38.065030 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:38.065033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:38.065492 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:38.065642 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:38.065651 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:38.066676 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:38.066679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:38.067719 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:38.067733 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:38.068377 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:38.068381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:38.068513 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:38.069055 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:38.069072 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:38.071818 (+ 2746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":339,"threads_started":1}
W20260811 00:13:38.072259 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:38.064159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:57257 (local address 127.7.94.62:35503)
0811 00:13:38.064373 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:38.064378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:38.064704 (+ 326us) server_negotiation.cc:407] Connection header received
0811 00:13:38.064838 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:38.064842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:38.064902 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:38.064983 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:38.065770 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:38.066551 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:38.067839 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:38.068254 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:38.068347 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:38.072018 (+ 3671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.1:57257: BlockingRecv error: recv got EOF from 127.7.94.1:57257 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 00:13:38 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:38 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:38.568668 23427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:38.560326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.2:37601)
0811 00:13:38.560839 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:38.560856 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:38.560992 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:38.561303 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:38.561307 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:38.561688 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:38.561839 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:38.561848 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:38.562813 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:38.562817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:38.563770 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:38.563781 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:38.564507 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:38.564512 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:38.564724 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:38.565449 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:38.565467 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:38.568437 (+ 2970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":269,"threads_started":1}
W20260811 00:13:38.568917 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:38.560431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:37601 (local address 127.7.94.62:35503)
0811 00:13:38.560683 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:38.560689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:38.560936 (+ 247us) server_negotiation.cc:407] Connection header received
0811 00:13:38.561088 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:38.561092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:38.561154 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:38.561282 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:38.561960 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:38.562701 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:38.563916 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:38.564353 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:38.564461 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:38.568639 (+ 4178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.2:37601: BlockingRecv error: recv got EOF from 127.7.94.2:37601 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 11 00:13:39 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:39 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:39.035434 23428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:39.023544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.3:53225)
0811 00:13:39.024112 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:39.024130 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:39.024244 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:39.024523 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:39.024526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:39.024888 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:39.025038 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:39.025046 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:39.026227 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:39.026231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:39.027183 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:39.027195 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:39.027831 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:39.027835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:39.027965 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:39.032165 (+ 4200us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:39.032183 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:39.035122 (+ 2939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":306,"threads_started":1}
W20260811 00:13:39.035619 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:39.023569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:53225 (local address 127.7.94.62:35503)
0811 00:13:39.023760 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:39.023765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:39.024206 (+ 441us) server_negotiation.cc:407] Connection header received
0811 00:13:39.024341 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:39.024344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:39.024404 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:39.024482 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:39.025162 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:39.026106 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:39.027306 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:39.027713 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:39.027808 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:39.035343 (+ 7535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.3:53225: BlockingRecv error: recv got EOF from 127.7.94.3:53225 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 00:13:39 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:39 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:39.081076 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:39.073019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.1:60091)
0811 00:13:39.073590 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:39.073608 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:39.073723 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:39.074007 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:39.074010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:39.074449 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:39.074614 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:39.074623 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:39.075568 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:39.075571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:39.076503 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:39.076515 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:39.077154 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:39.077158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:39.077298 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:39.077865 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:39.077889 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:39.080700 (+ 2811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":300,"threads_started":1}
W20260811 00:13:39.081280 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:39.073068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:60091 (local address 127.7.94.62:35503)
0811 00:13:39.073292 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:39.073299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:39.073687 (+ 388us) server_negotiation.cc:407] Connection header received
0811 00:13:39.073819 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:39.073823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:39.073882 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:39.073962 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:39.074744 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:39.075451 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:39.076621 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:39.077020 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:39.077109 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:39.080984 (+ 3875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.1:60091: BlockingRecv error: recv got EOF from 127.7.94.1:60091 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 00:13:39 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:39 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:39.578689 23431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:39.569959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.2:38167)
0811 00:13:39.570548 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:39.570565 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:39.570679 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:39.570980 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:39.570983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:39.571458 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:39.571737 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:39.571751 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:39.573206 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:39.573235 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:39.574219 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:39.574233 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:39.574885 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:39.574890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:39.575015 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:39.575603 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:39.575620 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:39.578447 (+ 2827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":307,"threads_started":1}
W20260811 00:13:39.578876 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:39.570147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:38167 (local address 127.7.94.62:35503)
0811 00:13:39.570430 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:39.570435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:39.570643 (+ 208us) server_negotiation.cc:407] Connection header received
0811 00:13:39.570778 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:39.570782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:39.570841 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:39.570937 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:39.571915 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:39.573048 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:39.574345 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:39.574754 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:39.574854 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:39.578623 (+ 3769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.2:38167: BlockingRecv error: recv got EOF from 127.7.94.2:38167 (error 108)
Metrics: {"server-negotiator.queue_time_us":160}
Aug 11 00:13:40 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:40 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:40.044922 23432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:40.036647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.3:53443)
0811 00:13:40.037521 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:40.037538 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:40.037713 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:40.038117 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:40.038122 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:40.038583 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:40.038751 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:40.038760 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:40.039722 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:40.039726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:40.040628 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:40.040639 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:40.041318 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:40.041322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:40.041443 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:40.041993 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:40.042009 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:40.044684 (+ 2675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":746,"thread_start_us":587,"threads_started":1}
W20260811 00:13:40.045131 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:40.036760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:53443 (local address 127.7.94.62:35503)
0811 00:13:40.036991 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:40.036996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:40.037641 (+ 645us) server_negotiation.cc:407] Connection header received
0811 00:13:40.037846 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:40.037853 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:40.037949 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:40.038055 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:40.038879 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:40.039601 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:40.040750 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:40.041146 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:40.041318 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:40.044861 (+ 3543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.3:53443: BlockingRecv error: recv got EOF from 127.7.94.3:53443 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 00:13:40 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:40 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:40.090304 23433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:40.082126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.1:34239)
0811 00:13:40.082669 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:40.082686 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:40.082800 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:40.083096 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:40.083100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:40.083490 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:40.083651 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:40.083660 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:40.084680 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:40.084693 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:40.085789 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:40.085807 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:40.086596 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:40.086600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:40.086724 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:40.087290 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:40.087308 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:40.090080 (+ 2772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":307,"threads_started":1}
W20260811 00:13:40.090509 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:40.082212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:34239 (local address 127.7.94.62:35503)
0811 00:13:40.082377 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:40.082381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:40.082760 (+ 379us) server_negotiation.cc:407] Connection header received
0811 00:13:40.082894 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:40.082898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:40.082961 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:40.083052 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:40.083772 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:40.084503 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:40.085954 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:40.086439 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:40.086534 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:40.090264 (+ 3730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.1:34239: BlockingRecv error: recv got EOF from 127.7.94.1:34239 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 00:13:40 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:40 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:40.587983 23434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:40.579708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.2:41337)
0811 00:13:40.580264 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:40.580280 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:40.580438 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:40.580824 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:40.580829 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:40.581297 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:40.581446 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:40.581455 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:40.582504 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:40.582511 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:40.583408 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:40.583419 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:40.584141 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:40.584148 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:40.584325 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:40.584885 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:40.584902 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:40.587659 (+ 2757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":306,"threads_started":1}
W20260811 00:13:40.588186 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:40.579808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:41337 (local address 127.7.94.62:35503)
0811 00:13:40.580040 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:40.580048 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:40.580366 (+ 318us) server_negotiation.cc:407] Connection header received
0811 00:13:40.580565 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:40.580575 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:40.580665 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:40.580773 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:40.581629 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:40.582361 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:40.583558 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:40.583972 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:40.584109 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:40.587872 (+ 3763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.2:41337: BlockingRecv error: recv got EOF from 127.7.94.2:41337 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 00:13:41 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:41 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:41.054358 23435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:41.045982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.3:59575)
0811 00:13:41.046613 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:41.046629 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:41.046803 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:41.047161 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:41.047166 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:41.047573 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:41.047734 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:41.047742 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:41.048738 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:41.048743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:41.049727 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:41.049738 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:41.050509 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:41.050515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:41.050698 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:41.051285 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:41.051302 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:41.054026 (+ 2724us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":359,"threads_started":1}
W20260811 00:13:41.054550 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:41.046049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:59575 (local address 127.7.94.62:35503)
0811 00:13:41.046236 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:41.046242 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:41.046717 (+ 475us) server_negotiation.cc:407] Connection header received
0811 00:13:41.046926 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:41.046931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:41.047017 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:41.047128 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:41.047877 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:41.048599 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:41.049892 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:41.050349 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:41.050493 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:41.054290 (+ 3797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.3:59575: BlockingRecv error: recv got EOF from 127.7.94.3:59575 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 00:13:41 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:41 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:41.099990 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:41.091338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.1:38781)
0811 00:13:41.091992 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:41.092008 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:41.092183 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:41.092529 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:41.092534 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:41.092981 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:41.093175 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:41.093184 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:41.094550 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:41.094554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:41.095527 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:41.095539 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:41.096189 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:41.096193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:41.096305 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:41.096860 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:41.096877 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:41.099736 (+ 2859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":279,"threads_started":1}
W20260811 00:13:41.100128 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:41.091495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:38781 (local address 127.7.94.62:35503)
0811 00:13:41.091744 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:41.091752 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:41.092097 (+ 345us) server_negotiation.cc:407] Connection header received
0811 00:13:41.092331 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:41.092335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:41.092394 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:41.092493 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:41.093365 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:41.094379 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:41.095649 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:41.096044 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:41.096135 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:41.099932 (+ 3797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.1:38781: BlockingRecv error: recv got EOF from 127.7.94.1:38781 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 00:13:41 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:41 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:41.597831 23440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:41.589178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.2:56589)
0811 00:13:41.589810 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:41.589826 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:41.589939 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:41.590220 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:41.590223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:41.590778 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:41.590951 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:41.590960 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:41.592334 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:41.592338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:41.593301 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:41.593313 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:41.593919 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:41.593923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:41.594033 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:41.594556 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:41.594573 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:41.597454 (+ 2881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":237,"threads_started":1}
W20260811 00:13:41.597903 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:41.589242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:56589 (local address 127.7.94.62:35503)
0811 00:13:41.589446 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:41.589453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:41.589911 (+ 458us) server_negotiation.cc:407] Connection header received
0811 00:13:41.590035 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:41.590039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:41.590098 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:41.590174 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:41.591089 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:41.592198 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:41.593420 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:41.593798 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:41.593887 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:41.597640 (+ 3753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.2:56589: BlockingRecv error: recv got EOF from 127.7.94.2:56589 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 00:13:42 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:42 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:42.064407 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:42.055669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.3:40721)
0811 00:13:42.056248 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:42.056266 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:42.056449 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:42.056843 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:42.056849 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:42.057479 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:42.057733 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:42.057747 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:42.058893 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:42.058897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:42.059808 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:42.059819 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:42.060479 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:42.060483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:42.060592 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:42.061144 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:42.061160 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:42.064038 (+ 2878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":230,"threads_started":1}
W20260811 00:13:42.064499 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:42.055718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:40721 (local address 127.7.94.62:35503)
0811 00:13:42.055935 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:42.055944 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:42.056393 (+ 449us) server_negotiation.cc:407] Connection header received
0811 00:13:42.056577 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:42.056585 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:42.056681 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:42.056792 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:42.057900 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:42.058766 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:42.059930 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:42.060346 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:42.060436 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:42.064264 (+ 3828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.3:40721: BlockingRecv error: recv got EOF from 127.7.94.3:40721 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 00:13:42 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:42 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:42.109496 23444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:42.101032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.1:52631)
0811 00:13:42.101564 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:42.101585 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:42.101715 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:42.102001 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:42.102004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:42.102396 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:42.102558 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:42.102567 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:42.103554 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:42.103558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:42.104746 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:42.104759 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:42.105877 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:42.105882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:42.106000 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:42.106550 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:42.106568 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:42.109200 (+ 2632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":390,"threads_started":1}
W20260811 00:13:42.109674 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:42.101305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:52631 (local address 127.7.94.62:35503)
0811 00:13:42.101513 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:42.101520 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:42.101679 (+ 159us) server_negotiation.cc:407] Connection header received
0811 00:13:42.101812 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:42.101816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:42.101876 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:42.101955 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:42.102692 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:42.103417 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:42.104979 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:42.105704 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:42.105801 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:42.109456 (+ 3655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.1:52631: BlockingRecv error: recv got EOF from 127.7.94.1:52631 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 00:13:42 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:42 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:42.607622 23445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:42.598877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.2:34703)
0811 00:13:42.599399 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:42.599415 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:42.599528 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:42.599873 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:42.599877 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:42.600277 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:42.600423 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:42.600431 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:42.601627 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:42.601631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:42.602530 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:42.602540 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:42.603487 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:42.603493 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:42.603699 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:42.604475 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:42.604493 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:42.607403 (+ 2910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":297,"threads_started":1}
W20260811 00:13:42.607800 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:42.599003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:34703 (local address 127.7.94.62:35503)
0811 00:13:42.599170 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:42.599174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:42.599503 (+ 329us) server_negotiation.cc:407] Connection header received
0811 00:13:42.599669 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:42.599673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:42.599732 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:42.599810 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:42.600556 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:42.601486 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:42.602687 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:42.603270 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:42.603420 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:42.607590 (+ 4170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.2:34703: BlockingRecv error: recv got EOF from 127.7.94.2:34703 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 00:13:43 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:43 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:43.074460 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:43.065642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.3:47797)
0811 00:13:43.066265 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:43.066281 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:43.066397 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:43.066791 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:43.066799 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:43.067255 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:43.067417 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:43.067425 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:43.068648 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:43.068653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:43.069588 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:43.069600 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:43.070340 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:43.070347 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:43.070508 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:43.071202 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:43.071220 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:43.074230 (+ 3010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":358,"threads_started":1}
W20260811 00:13:43.074672 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:43.065656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:47797 (local address 127.7.94.62:35503)
0811 00:13:43.065844 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:43.065848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:43.066374 (+ 526us) server_negotiation.cc:407] Connection header received
0811 00:13:43.066499 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:43.066503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:43.066584 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:43.066706 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:43.067561 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:43.068502 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:43.069701 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:43.070116 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:43.070207 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:43.074426 (+ 4219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.3:47797: BlockingRecv error: recv got EOF from 127.7.94.3:47797 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 00:13:43 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:43 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:43.118798 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:43.110408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.1:56777)
0811 00:13:43.111010 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:43.111026 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:43.111159 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:43.111455 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:43.111458 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:43.111864 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:43.112051 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:43.112061 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:43.113017 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:43.113021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:43.114011 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:43.114024 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:43.114681 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:43.114685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:43.114836 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:43.115402 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:43.115419 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:43.118414 (+ 2995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":336,"threads_started":1}
W20260811 00:13:43.118965 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:43.110451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:56777 (local address 127.7.94.62:35503)
0811 00:13:43.110609 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:43.110614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:43.111097 (+ 483us) server_negotiation.cc:407] Connection header received
0811 00:13:43.111256 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:43.111260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:43.111319 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:43.111403 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:43.112193 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:43.112906 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:43.114130 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:43.114546 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:43.114647 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:43.118736 (+ 4089us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.1:56777: BlockingRecv error: recv got EOF from 127.7.94.1:56777 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 00:13:43 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:43 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:43.616700 23449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:43.608719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.2:51639)
0811 00:13:43.609292 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:43.609308 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:43.609475 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:43.609800 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:43.609804 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:43.610204 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:43.610371 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:43.610380 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:43.611374 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:43.611378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:43.612305 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:43.612316 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:43.612954 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:43.612958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:43.613076 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:43.613719 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:43.613737 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:43.616463 (+ 2726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":289,"threads_started":1}
W20260811 00:13:43.616902 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:43.608772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:51639 (local address 127.7.94.62:35503)
0811 00:13:43.609005 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:43.609011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:43.609382 (+ 371us) server_negotiation.cc:407] Connection header received
0811 00:13:43.609610 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:43.609614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:43.609675 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:43.609753 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:43.610500 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:43.611258 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:43.612422 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:43.612819 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:43.612906 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:43.616659 (+ 3753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.2:51639: BlockingRecv error: recv got EOF from 127.7.94.2:51639 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 00:13:44 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:44 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:44.087463 23450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:44.075506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.3:40723)
0811 00:13:44.075975 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:44.075991 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:44.076121 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:44.076409 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:44.076412 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:44.076798 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:44.076970 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:44.076977 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:44.078005 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:44.078009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:44.078936 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:44.078948 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:44.079608 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:44.079613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:44.079730 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:44.084222 (+ 4492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:44.084250 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:44.087224 (+ 2974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":249,"threads_started":1}
W20260811 00:13:44.087749 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:44.075590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:40723 (local address 127.7.94.62:35503)
0811 00:13:44.075846 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:44.075852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:44.076078 (+ 226us) server_negotiation.cc:407] Connection header received
0811 00:13:44.076220 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:44.076223 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:44.076286 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:44.076366 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:44.077091 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:44.077889 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:44.079055 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:44.079474 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:44.079563 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:44.087444 (+ 7881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.3:40723: BlockingRecv error: recv got EOF from 127.7.94.3:40723 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 11 00:13:44 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:44 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:44.127847 23451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:44.119887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.1:51763)
0811 00:13:44.120340 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:44.120356 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:44.120555 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:44.120853 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:44.120857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:44.121292 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:44.121446 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:44.121455 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:44.122460 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:44.122464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:44.123372 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:44.123383 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:44.124042 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:44.124045 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:44.124196 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:44.124767 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:44.124784 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:44.127625 (+ 2841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":265,"threads_started":1}
W20260811 00:13:44.128067 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:44.119984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:51763 (local address 127.7.94.62:35503)
0811 00:13:44.120236 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:44.120243 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:44.120468 (+ 225us) server_negotiation.cc:407] Connection header received
0811 00:13:44.120668 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:44.120671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:44.120731 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:44.120807 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:44.121598 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:44.122332 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:44.123488 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:44.123918 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:44.124005 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:44.127818 (+ 3813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.1:51763: BlockingRecv error: recv got EOF from 127.7.94.1:51763 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 11 00:13:44 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:44 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:44.626448 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:44.617751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.2:41987)
0811 00:13:44.618316 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:44.618331 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:44.618442 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:44.618734 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:44.618737 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:44.619125 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:44.619274 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:44.619282 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:44.620420 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:44.620423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:44.621539 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:44.621552 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:44.622227 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:44.622231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:44.622349 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:44.623027 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:44.623044 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:44.626112 (+ 3068us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":319,"threads_started":1}
W20260811 00:13:44.626724 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:44.617842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:41987 (local address 127.7.94.62:35503)
0811 00:13:44.618007 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:44.618017 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:44.618403 (+ 386us) server_negotiation.cc:407] Connection header received
0811 00:13:44.618544 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:44.618547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:44.618608 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:44.618690 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:44.619407 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:44.620294 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:44.621687 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:44.622097 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:44.622194 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:44.626422 (+ 4228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.2:41987: BlockingRecv error: recv got EOF from 127.7.94.2:41987 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 00:13:45 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:45 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:45.096704 23455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:45.088494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.3:53083)
0811 00:13:45.089068 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:45.089085 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:45.089195 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:45.089546 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:45.089549 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:45.089937 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:45.090087 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:45.090095 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:45.091090 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:45.091094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:45.092253 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:45.092266 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:45.092910 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:45.092915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:45.093027 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:45.093697 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:45.093713 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:45.096488 (+ 2775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":331,"threads_started":1}
W20260811 00:13:45.096907 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:45.088666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:53083 (local address 127.7.94.62:35503)
0811 00:13:45.088845 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:45.088850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:45.089160 (+ 310us) server_negotiation.cc:407] Connection header received
0811 00:13:45.089348 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:45.089351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:45.089410 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:45.089499 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:45.090226 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:45.090963 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:45.092377 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:45.092781 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:45.092873 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:45.096690 (+ 3817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.3:53083: BlockingRecv error: recv got EOF from 127.7.94.3:53083 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 00:13:45 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:45 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:45.137290 23456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:45.128883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.1:57647)
0811 00:13:45.129399 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:45.129415 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:45.129533 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:45.129835 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:45.129839 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:45.130238 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:45.130418 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:45.130426 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:45.131364 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:45.131368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:45.132270 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:45.132280 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:45.132967 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:45.132971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:45.133093 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:45.133862 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:45.133887 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:45.136992 (+ 3105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":253,"threads_started":1}
W20260811 00:13:45.137665 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:45.128943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:57647 (local address 127.7.94.62:35503)
0811 00:13:45.129177 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:45.129184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:45.129497 (+ 313us) server_negotiation.cc:407] Connection header received
0811 00:13:45.129632 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:45.129635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:45.129708 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:45.129788 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:45.130534 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:45.131248 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:45.132385 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:45.132813 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:45.132903 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:45.137193 (+ 4290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.1:57647: BlockingRecv error: recv got EOF from 127.7.94.1:57647 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 00:13:45 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:45 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:45.636093 23459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:45.627583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.2:45369)
0811 00:13:45.628100 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:45.628117 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:45.628280 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:45.628631 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:45.628636 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:45.629074 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:45.629323 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:45.629333 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:45.630429 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:45.630437 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:45.631362 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:45.631373 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:45.632086 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:45.632092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:45.632274 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:45.632920 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:45.632937 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:45.635738 (+ 2801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":277,"threads_started":1}
W20260811 00:13:45.636214 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:45.627659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:45369 (local address 127.7.94.62:35503)
0811 00:13:45.627897 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:45.627906 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:45.628208 (+ 302us) server_negotiation.cc:407] Connection header received
0811 00:13:45.628413 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:45.628418 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:45.628479 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:45.628583 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:45.629466 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:45.630287 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:45.631505 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:45.631929 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:45.632070 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:45.635942 (+ 3872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.2:45369: BlockingRecv error: recv got EOF from 127.7.94.2:45369 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 00:13:46 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:46 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:46.107841 23462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:46.097753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.3:33161)
0811 00:13:46.098355 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:46.098371 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:46.098482 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:46.098844 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:46.098847 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:46.099236 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:46.099415 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:46.099422 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:46.101202 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:46.101207 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:46.102556 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:46.102570 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:46.103758 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:46.103762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:46.103875 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:46.104669 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:46.104695 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:46.107601 (+ 2906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":378,"threads_started":1}
W20260811 00:13:46.108368 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:46.097887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:33161 (local address 127.7.94.62:35503)
0811 00:13:46.098127 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:46.098136 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:46.098454 (+ 318us) server_negotiation.cc:407] Connection header received
0811 00:13:46.098583 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:46.098591 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:46.098690 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:46.098819 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:46.099555 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:46.101046 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:46.102695 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:46.103618 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:46.103765 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:46.107889 (+ 4124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.3:33161: BlockingRecv error: recv got EOF from 127.7.94.3:33161 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 00:13:46 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:46 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:46.149330 23463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:46.138488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.1:38347)
0811 00:13:46.139191 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:46.139215 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:46.139367 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:46.139694 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:46.139701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:46.140285 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:46.140538 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:46.140553 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:46.141723 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:46.141731 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:46.143611 (+ 1880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:46.143632 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:46.144341 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:46.144350 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:46.144601 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:46.145314 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:46.145343 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:46.148957 (+ 3614us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":371,"threads_started":1}
W20260811 00:13:46.149726 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:46.138521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:38347 (local address 127.7.94.62:35503)
0811 00:13:46.138772 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:46.138781 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:46.139305 (+ 524us) server_negotiation.cc:407] Connection header received
0811 00:13:46.139482 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:46.139486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:46.139547 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:46.139623 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:46.140703 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:46.141546 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:46.143756 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:46.144194 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:46.144286 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:46.149156 (+ 4870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.1:38347: BlockingRecv error: recv got EOF from 127.7.94.1:38347 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 00:13:46 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:46 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:46.645293 23466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:46.637273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.2:35031)
0811 00:13:46.637776 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:46.637793 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:46.637905 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:46.638175 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:46.638177 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:46.638563 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:46.638712 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:46.638720 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:46.639626 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:46.639629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:46.640552 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:46.640563 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:46.641275 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:46.641282 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:46.641414 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:46.641986 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:46.642004 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:46.644968 (+ 2964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":285,"threads_started":1}
W20260811 00:13:46.645541 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:46.637332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:35031 (local address 127.7.94.62:35503)
0811 00:13:46.637576 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:46.637585 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:46.637873 (+ 288us) server_negotiation.cc:407] Connection header received
0811 00:13:46.637989 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:46.637992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:46.638054 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:46.638129 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:46.638832 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:46.639506 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:46.640682 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:46.641090 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:46.641177 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:46.645281 (+ 4104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.2:35031: BlockingRecv error: recv got EOF from 127.7.94.2:35031 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 00:13:47 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:47 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:47.117508 23467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:47.108790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.3:42277)
0811 00:13:47.109499 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:47.109536 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:47.109694 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:47.110224 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:47.110228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:47.110610 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:47.110756 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:47.110765 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:47.111803 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:47.111808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:47.112741 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:47.112752 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:47.113489 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:47.113493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:47.113631 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:47.114299 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:47.114317 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:47.117253 (+ 2936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":446,"threads_started":1}
W20260811 00:13:47.117764 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:47.108835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:42277 (local address 127.7.94.62:35503)
0811 00:13:47.109092 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:47.109101 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:47.109621 (+ 520us) server_negotiation.cc:407] Connection header received
0811 00:13:47.109832 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:47.109837 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:47.109930 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:47.110068 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:47.110891 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:47.111645 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:47.112867 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:47.113361 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:47.113457 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:47.117476 (+ 4019us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.3:42277: BlockingRecv error: recv got EOF from 127.7.94.3:42277 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 11 00:13:47 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:47 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:47.159138 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:47.150274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.1:45155)
0811 00:13:47.150801 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:47.150816 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:47.150977 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:47.151331 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:47.151335 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:47.151821 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:47.152000 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:47.152009 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:47.153041 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:47.153046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:47.154039 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:47.154051 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:47.154764 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:47.154770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:47.154945 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:47.155508 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:47.155525 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:47.158874 (+ 3349us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":263,"threads_started":1}
W20260811 00:13:47.159547 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:47.150351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:45155 (local address 127.7.94.62:35503)
0811 00:13:47.150594 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:47.150600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:47.150906 (+ 306us) server_negotiation.cc:407] Connection header received
0811 00:13:47.151116 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:47.151121 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:47.151193 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:47.151296 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:47.152164 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:47.152895 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:47.154188 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:47.154610 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:47.154755 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:47.159120 (+ 4365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.1:45155: BlockingRecv error: recv got EOF from 127.7.94.1:45155 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 00:13:47 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:47 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:47.655447 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:47.646551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.2:50681)
0811 00:13:47.647335 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:47.647360 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:47.647514 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:47.647970 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:47.647975 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:47.648574 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:47.648853 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:47.648870 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:47.649924 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:47.649928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:47.650880 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:47.650890 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:47.651570 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:47.651574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:47.651686 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:47.652269 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:47.652285 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:47.655109 (+ 2824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":553,"threads_started":1}
W20260811 00:13:47.655553 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:47.646933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:50681 (local address 127.7.94.62:35503)
0811 00:13:47.647106 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:47.647110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:47.647474 (+ 364us) server_negotiation.cc:407] Connection header received
0811 00:13:47.647645 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:47.647652 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:47.647759 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:47.647874 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:47.649025 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:47.649790 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:47.651010 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:47.651417 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:47.651507 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:47.655313 (+ 3806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.2:50681: BlockingRecv error: recv got EOF from 127.7.94.2:50681 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 00:13:48 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:48 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:48.127264 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:48.118441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.3:47439)
0811 00:13:48.118995 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:48.119011 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:48.119129 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:48.119425 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:48.119428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:48.119849 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:48.120029 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:48.120038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:48.121168 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:48.121172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:48.122443 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:48.122457 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:48.123415 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:48.123420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:48.123533 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:48.124117 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:48.124136 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:48.127007 (+ 2871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":292,"threads_started":1}
W20260811 00:13:48.127436 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:48.118526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:47439 (local address 127.7.94.62:35503)
0811 00:13:48.118721 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:48.118725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:48.119084 (+ 359us) server_negotiation.cc:407] Connection header received
0811 00:13:48.119239 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:48.119243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:48.119300 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:48.119379 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:48.120203 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:48.121029 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:48.122588 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:48.123240 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:48.123347 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:48.127224 (+ 3877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.3:47439: BlockingRecv error: recv got EOF from 127.7.94.3:47439 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 00:13:48 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:48 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:48.168776 23475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:48.160083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.1:59521)
0811 00:13:48.160586 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:48.160612 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:48.160724 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:48.161025 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:48.161028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:48.161462 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:48.161624 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:48.161633 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:48.162705 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:48.162709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:48.163614 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:48.163625 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:48.164337 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:48.164341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:48.164457 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:48.165071 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:48.165087 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:48.168395 (+ 3308us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":266,"threads_started":1}
W20260811 00:13:48.168897 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:48.160195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:59521 (local address 127.7.94.62:35503)
0811 00:13:48.160593 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:48.160598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:48.160683 (+ 85us) server_negotiation.cc:407] Connection header received
0811 00:13:48.160828 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:48.160832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:48.160897 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:48.160983 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:48.161767 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:48.162586 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:48.163762 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:48.164186 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:48.164295 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:48.168635 (+ 4340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.1:59521: BlockingRecv error: recv got EOF from 127.7.94.1:59521 (error 108)
Metrics: {"server-negotiator.queue_time_us":217}
Aug 11 00:13:48 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:48 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:48.665151 23476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:48.656533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.2:38179)
0811 00:13:48.657061 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:48.657078 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:48.657196 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:48.657516 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:48.657519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:48.657916 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:48.658070 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:48.658078 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:48.659050 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:48.659054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:48.659973 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:48.659983 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:48.660677 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:48.660681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:48.660797 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:48.661572 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:48.661618 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:48.664922 (+ 3304us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":287,"threads_started":1}
W20260811 00:13:48.665421 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:48.656815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:38179 (local address 127.7.94.62:35503)
0811 00:13:48.657025 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:48.657032 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:48.657164 (+ 132us) server_negotiation.cc:407] Connection header received
0811 00:13:48.657321 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:48.657325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:48.657387 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:48.657466 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:48.658212 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:48.658927 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:48.660091 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:48.660541 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:48.660635 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:48.665122 (+ 4487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.2:38179: BlockingRecv error: recv got EOF from 127.7.94.2:38179 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 11 00:13:49 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:49 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.3@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:49.144716 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:49.128171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.3:58793)
0811 00:13:49.128737 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:49.128773 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:49.128927 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:49.129334 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:49.129341 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:49.129894 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:49.130109 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:49.130140 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:49.131327 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:49.131331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:49.132246 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:49.132257 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:49.132939 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:49.132945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:49.133153 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:49.141157 (+ 8004us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:49.141176 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:49.144466 (+ 3290us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":301,"threads_started":1}
W20260811 00:13:49.144939 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:49.128369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:58793 (local address 127.7.94.62:35503)
0811 00:13:49.128591 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:49.128613 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:49.128874 (+ 261us) server_negotiation.cc:407] Connection header received
0811 00:13:49.129092 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:49.129097 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:49.129178 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:49.129315 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:49.130277 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:49.131194 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:49.132373 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:49.132774 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:49.132867 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:49.144705 (+ 11838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.3:58793: BlockingRecv error: recv got EOF from 127.7.94.3:58793 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 00:13:49 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:49 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.1@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:49.178494 23482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:49.170097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.1:35261)
0811 00:13:49.170696 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:49.170712 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:49.170821 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:49.171140 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:49.171144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:49.171661 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:49.171846 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:49.171856 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:49.172872 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:49.172876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:49.173807 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:49.173819 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:49.174497 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:49.174501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:49.174606 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:49.175242 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:49.175259 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:49.178139 (+ 2880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":314,"threads_started":1}
W20260811 00:13:49.178610 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:49.170143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:35261 (local address 127.7.94.62:35503)
0811 00:13:49.170313 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:49.170317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:49.170781 (+ 464us) server_negotiation.cc:407] Connection header received
0811 00:13:49.170914 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:49.170917 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:49.170974 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:49.171054 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:49.171989 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:49.172744 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:49.173940 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:49.174363 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:49.174458 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:49.178359 (+ 3901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.1:35261: BlockingRecv error: recv got EOF from 127.7.94.1:35261 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 00:13:49 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 00:13:49 dist-test-slave-8hhm krb5kdc[21656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.94.2@KRBTEST.COM for oryx/127.7.94.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 00:13:49.675292 23485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:49.666467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35503 (local address 127.7.94.2:36501)
0811 00:13:49.666982 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:49.667024 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:49.667211 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:49.667555 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:49.667560 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:49.668061 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:49.668238 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:49.668248 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:49.669337 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:49.669342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:49.670232 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:49.670242 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:49.671141 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:49.671149 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:49.671342 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:49.672132 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:49.672166 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:49.675034 (+ 2868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.94.62:35503: Server oryx/127.7.94.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":310,"threads_started":1}
W20260811 00:13:49.675442 22913 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:49.666533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:36501 (local address 127.7.94.62:35503)
0811 00:13:49.666739 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:49.666748 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:49.667129 (+ 381us) server_negotiation.cc:407] Connection header received
0811 00:13:49.667337 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:49.667343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:49.667418 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:49.667519 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:49.668382 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:49.669168 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:49.670368 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:49.670945 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:49.671082 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:49.675211 (+ 4129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.94.2:36501: BlockingRecv error: recv got EOF from 127.7.94.2:36501 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 00:13:49.966872 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 22654
I20260811 00:13:49.978663 23492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:49.967832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56090 (local address 127.7.94.1:46517)
0811 00:13:49.968445 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:49.968452 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:49.968479 (+ 27us) server_negotiation.cc:407] Connection header received
0811 00:13:49.968538 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:49.968544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:49.968785 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:49.968939 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:49.970295 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:49.971428 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:49.972746 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:49.973349 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:49.973445 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:49.975882 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:49.975889 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:49.975903 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:49.975931 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:49.978229 (+ 2298us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.94.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 00:13:49.978452 (+ 223us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56090: Request ticket server kudu/127.7.94.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":305,"threads_started":1}
W20260811 00:13:49.978786 23492 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56090: Request ticket server kudu/127.7.94.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 00:13:49.978878 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 22654: Not authorized: Client connection negotiation failed: client connection to 127.7.94.1:46517: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.94.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 00:13:49.979074 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 22654
I20260811 00:13:50.007375 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 22916
I20260811 00:13:50.019184 23493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:50.008438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40314 (local address 127.7.94.2:32895)
0811 00:13:50.009065 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:50.009072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:50.009105 (+ 33us) server_negotiation.cc:407] Connection header received
0811 00:13:50.009168 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:50.009174 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:50.009448 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:50.009613 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:50.010829 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:50.011915 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:50.013044 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:50.013649 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:50.013745 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:50.016174 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:50.016181 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:50.016195 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:50.016228 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:50.018678 (+ 2450us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.94.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 00:13:50.018909 (+ 231us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40314: Request ticket server kudu/127.7.94.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":304,"threads_started":1}
W20260811 00:13:50.019294 23493 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40314: Request ticket server kudu/127.7.94.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 00:13:50.019398 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 22916: Not authorized: Client connection negotiation failed: client connection to 127.7.94.2:32895: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.94.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 00:13:50.019539 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 22916
I20260811 00:13:50.042646 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 23162
I20260811 00:13:50.054273 23495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:50.043626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36140 (local address 127.7.94.3:34261)
0811 00:13:50.044206 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:50.044213 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:50.044241 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:13:50.044303 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:50.044310 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:50.044532 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:50.044713 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:50.046057 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:50.047037 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:50.048139 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:50.048667 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:50.048770 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:50.051179 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:50.051187 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:50.051200 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:50.051228 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:50.053826 (+ 2598us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.94.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 00:13:50.054053 (+ 227us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36140: Request ticket server kudu/127.7.94.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":295,"threads_started":1}
W20260811 00:13:50.054394 23495 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36140: Request ticket server kudu/127.7.94.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 00:13:50.054380 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 23162: Not authorized: Client connection negotiation failed: client connection to 127.7.94.3:34261: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.94.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 00:13:50.054533 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 23162
I20260811 00:13:50.081324 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 22536
I20260811 00:13:50.082573 22641 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:50.200836 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 22536
2026-08-11T00:13:50Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21627 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T00:13:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:13:50Z Disabled control of system clock
I20260811 00:13:50.297202 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:34001
--webserver_interface=127.7.94.62
--webserver_port=0
--builtin_ntp_servers=127.7.94.20:45383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:34001
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 00:13:50.619236 23508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:50.619561 23508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:50.619725 23508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:50.626716 23508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:13:50.626858 23508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:50.626950 23508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:50.627059 23508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:13:50.627146 23508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:13:50.635824 23508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:45383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:34001
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:34001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:50.637046 23508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:50.638832 23508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:50.650662 23526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:50.650769 23518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:50.650748 23520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:50.650986 23508 server_base.cc:1034] running on GCE node
I20260811 00:13:50.652122 23508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:50.653414 23508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:50.654662 23508 hybrid_clock.cc:648] HybridClock initialized: now 1786407230654619 us; error 31 us; skew 500 ppm
I20260811 00:13:50.655027 23508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:13:50.657377 23508 webserver.cc:460] Webserver started at http://127.7.94.62:38665/ using document root <none> and password file <none>
I20260811 00:13:50.658062 23508 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:50.658216 23508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:50.658552 23508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:50.661435 23508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "fbbd791724fe4ec090b474dfd50d4b9e"
format_stamp: "Formatted at 2026-08-11 00:13:50 on dist-test-slave-8hhm"
server_key: "be99b1326d9ecce78748834a8468212e"
server_key_iv: "ee122023a06cd0fcc3396eae7fcb65d2"
server_key_version: "encryptionkey@0"
I20260811 00:13:50.662228 23508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "fbbd791724fe4ec090b474dfd50d4b9e"
format_stamp: "Formatted at 2026-08-11 00:13:50 on dist-test-slave-8hhm"
server_key: "be99b1326d9ecce78748834a8468212e"
server_key_iv: "ee122023a06cd0fcc3396eae7fcb65d2"
server_key_version: "encryptionkey@0"
I20260811 00:13:50.667488 23508 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:13:50.670728 23537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:50.671945 23508 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 00:13:50.672127 23508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "fbbd791724fe4ec090b474dfd50d4b9e"
format_stamp: "Formatted at 2026-08-11 00:13:50 on dist-test-slave-8hhm"
server_key: "be99b1326d9ecce78748834a8468212e"
server_key_iv: "ee122023a06cd0fcc3396eae7fcb65d2"
server_key_version: "encryptionkey@0"
I20260811 00:13:50.672338 23508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:50.686293 23508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:50.687088 23508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:50.687464 23508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:50.705569 23508 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:34001
I20260811 00:13:50.705587 23613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:34001 every 8 connection(s)
I20260811 00:13:50.706822 23508 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:13:50.708830 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 23508
I20260811 00:13:50.709026 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:13:50.709699 7544 external_mini_cluster.cc:1428] Setting key 94b39b1847b4e6cdad62a960ae420b04
I20260811 00:13:50.710659 23614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:50.725281 23617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:50.711729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38144 (local address 127.7.94.62:34001)
0811 00:13:50.712779 (+ 1050us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:50.712793 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:50.712841 (+ 48us) server_negotiation.cc:407] Connection header received
0811 00:13:50.715323 (+ 2482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:50.715352 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:50.715919 (+ 567us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:50.716356 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:50.716880 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:50.718859 (+ 1979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:50.720318 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:50.721340 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:50.721518 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:50.722624 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:50.722635 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:50.722652 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:13:50.723373 (+ 721us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:50.723476 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:13:50.723504 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:50.723618 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:50.723781 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 00:13:50.724069 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":346,"threads_started":1}
I20260811 00:13:50.725764 23614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e: Bootstrap starting.
I20260811 00:13:50.728816 23614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:50.729875 23614 log.cc:824] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e: Log is configured to *not* fsync() on all Append() calls
I20260811 00:13:50.732626 23614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e: No bootstrap required, opened a new log
I20260811 00:13:50.740082 23614 raft_consensus.cc:348] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbbd791724fe4ec090b474dfd50d4b9e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34001 } }
I20260811 00:13:50.740422 23614 raft_consensus.cc:374] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:50.740527 23614 raft_consensus.cc:729] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbbd791724fe4ec090b474dfd50d4b9e, State: Initialized, Role: FOLLOWER
I20260811 00:13:50.741021 23614 consensus_queue.cc:260] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbbd791724fe4ec090b474dfd50d4b9e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34001 } }
I20260811 00:13:50.741294 23614 raft_consensus.cc:388] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:13:50.741437 23614 raft_consensus.cc:482] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:13:50.741653 23614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:50.744108 23614 raft_consensus.cc:504] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbbd791724fe4ec090b474dfd50d4b9e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34001 } }
I20260811 00:13:50.744565 23614 leader_election.cc:302] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fbbd791724fe4ec090b474dfd50d4b9e; no voters:
I20260811 00:13:50.745097 23614 leader_election.cc:288] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:13:50.745291 23622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:13:50.746028 23622 raft_consensus.cc:686] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e [term 1 LEADER]: Becoming Leader. State: Replica: fbbd791724fe4ec090b474dfd50d4b9e, State: Running, Role: LEADER
I20260811 00:13:50.746546 23622 consensus_queue.cc:237] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbbd791724fe4ec090b474dfd50d4b9e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34001 } }
I20260811 00:13:50.747125 23614 sys_catalog.cc:564] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:13:50.749418 23628 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader fbbd791724fe4ec090b474dfd50d4b9e. Latest consensus state: current_term: 1 leader_uuid: "fbbd791724fe4ec090b474dfd50d4b9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbbd791724fe4ec090b474dfd50d4b9e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34001 } } }
I20260811 00:13:50.749886 23628 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:50.750504 23625 sys_catalog.cc:455] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fbbd791724fe4ec090b474dfd50d4b9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fbbd791724fe4ec090b474dfd50d4b9e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34001 } } }
I20260811 00:13:50.750938 23625 sys_catalog.cc:458] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:50.751730 23637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:13:50.756608 23637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:13:50.765340 23637 catalog_manager.cc:1347] Generated new cluster ID: d725bfb957674ac1b01372473a4b0068
I20260811 00:13:50.765458 23637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:13:50.800514 23637 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:13:50.801976 23637 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:13:50.809286 23637 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fbbd791724fe4ec090b474dfd50d4b9e: Generated new TSK 0
I20260811 00:13:50.810073 23637 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:13:50.822026 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:34001
--builtin_ntp_servers=127.7.94.20:45383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 00:13:51.177927 23655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:51.178258 23655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:51.178422 23655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:51.185508 23655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:51.185647 23655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:51.185815 23655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:13:51.195060 23655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:45383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:34001
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:51.196352 23655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:51.198194 23655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:51.211313 23673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:51.211444 23671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:51.211483 23678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:51.212491 23655 server_base.cc:1034] running on GCE node
I20260811 00:13:51.213047 23655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:51.213979 23655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:51.215191 23655 hybrid_clock.cc:648] HybridClock initialized: now 1786407231215134 us; error 44 us; skew 500 ppm
I20260811 00:13:51.215507 23655 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:13:51.217777 23655 webserver.cc:460] Webserver started at http://127.7.94.1:37319/ using document root <none> and password file <none>
I20260811 00:13:51.218487 23655 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:51.218604 23655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:51.218941 23655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:51.221812 23655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "92cf87511996405f8f1dc6c8be3f5cf3"
format_stamp: "Formatted at 2026-08-11 00:13:51 on dist-test-slave-8hhm"
server_key: "a1ba8777815db7c138d2a7470e2c74ad"
server_key_iv: "2ff4b3e16b18233c9187eb14ff184fdb"
server_key_version: "encryptionkey@0"
I20260811 00:13:51.222595 23655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "92cf87511996405f8f1dc6c8be3f5cf3"
format_stamp: "Formatted at 2026-08-11 00:13:51 on dist-test-slave-8hhm"
server_key: "a1ba8777815db7c138d2a7470e2c74ad"
server_key_iv: "2ff4b3e16b18233c9187eb14ff184fdb"
server_key_version: "encryptionkey@0"
I20260811 00:13:51.227941 23655 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:13:51.231278 23691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:51.232585 23655 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:13:51.232758 23655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "92cf87511996405f8f1dc6c8be3f5cf3"
format_stamp: "Formatted at 2026-08-11 00:13:51 on dist-test-slave-8hhm"
server_key: "a1ba8777815db7c138d2a7470e2c74ad"
server_key_iv: "2ff4b3e16b18233c9187eb14ff184fdb"
server_key_version: "encryptionkey@0"
I20260811 00:13:51.232969 23655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:51.249958 23655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:51.250780 23655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:51.251039 23655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:51.252127 23655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:51.253671 23655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:51.253820 23655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:51.253953 23655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:51.254056 23655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:51.287599 23655 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:35441
I20260811 00:13:51.287649 23890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:35441 every 8 connection(s)
I20260811 00:13:51.289048 23655 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:13:51.296079 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 23655
I20260811 00:13:51.296277 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:13:51.297164 7544 external_mini_cluster.cc:1428] Setting key 8b90ad5dab779deb12f88d6d24065e87
I20260811 00:13:51.300290 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:34001
--builtin_ntp_servers=127.7.94.20:45383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 00:13:51.303067 23895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:51.291735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:59519 (local address 127.7.94.62:34001)
0811 00:13:51.292253 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:51.292260 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:51.293493 (+ 1233us) server_negotiation.cc:407] Connection header received
0811 00:13:51.295267 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:51.295272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:51.295343 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:51.295441 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:51.296626 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:51.297786 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:51.299849 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:51.300557 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:51.300705 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:51.302078 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:51.302087 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:51.302094 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:13:51.302143 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:51.302196 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:13:51.302217 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:51.302315 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:51.302520 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 00:13:51.302630 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":259,"threads_started":1}
I20260811 00:13:51.303900 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:51.292359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:34001 (local address 127.7.94.1:59519)
0811 00:13:51.293338 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:51.293376 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:51.295002 (+ 1626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:51.295655 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:51.295663 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:51.295748 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:51.296440 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:51.296462 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:51.297971 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:51.297976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:51.299691 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:51.299709 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:51.300765 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:51.300770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:51.300963 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:51.301740 (+ 777us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:13:51.301743 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:51.301783 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:13:51.301864 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:51.302318 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:51.302326 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:51.302332 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 00:13:51.302477 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 00:13:51.302782 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":288,"threads_started":1}
I20260811 00:13:51.305552 23891 heartbeater.cc:347] Connected to a master server at 127.7.94.62:34001
I20260811 00:13:51.305876 23891 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:51.306638 23891 heartbeater.cc:511] Master 127.7.94.62:34001 requested a full tablet report, sending...
I20260811 00:13:51.308817 23564 ts_manager.cc:194] Registered new tserver with Master: 92cf87511996405f8f1dc6c8be3f5cf3 (127.7.94.1:35441)
I20260811 00:13:51.310510 23564 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.94.1:59519
W20260811 00:13:51.636783 23897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:51.637041 23897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:51.637123 23897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:51.644030 23897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:51.644121 23897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:51.644219 23897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:13:51.653575 23897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:45383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:34001
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:51.654980 23897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:51.656836 23897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:51.669703 23906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:51.669625 23907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:51.669654 23909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:51.670444 23897 server_base.cc:1034] running on GCE node
I20260811 00:13:51.671165 23897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:51.672068 23897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:51.673336 23897 hybrid_clock.cc:648] HybridClock initialized: now 1786407231673292 us; error 51 us; skew 500 ppm
I20260811 00:13:51.673667 23897 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:13:51.675904 23897 webserver.cc:460] Webserver started at http://127.7.94.2:36149/ using document root <none> and password file <none>
I20260811 00:13:51.676688 23897 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:51.676811 23897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:51.677138 23897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:51.679970 23897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "a4cc7c491f464dfebee1416bd8767bb7"
format_stamp: "Formatted at 2026-08-11 00:13:51 on dist-test-slave-8hhm"
server_key: "f080128eec571e83c67b05b143441d77"
server_key_iv: "572942408622959884564773c4b94ab2"
server_key_version: "encryptionkey@0"
I20260811 00:13:51.680755 23897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "a4cc7c491f464dfebee1416bd8767bb7"
format_stamp: "Formatted at 2026-08-11 00:13:51 on dist-test-slave-8hhm"
server_key: "f080128eec571e83c67b05b143441d77"
server_key_iv: "572942408622959884564773c4b94ab2"
server_key_version: "encryptionkey@0"
I20260811 00:13:51.685678 23897 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 00:13:51.688900 23925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:51.690418 23897 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 00:13:51.690615 23897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "a4cc7c491f464dfebee1416bd8767bb7"
format_stamp: "Formatted at 2026-08-11 00:13:51 on dist-test-slave-8hhm"
server_key: "f080128eec571e83c67b05b143441d77"
server_key_iv: "572942408622959884564773c4b94ab2"
server_key_version: "encryptionkey@0"
I20260811 00:13:51.690835 23897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:51.725088 23897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:51.726140 23897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:51.726456 23897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:51.732630 23897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:51.734673 23897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:51.734823 23897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:51.734954 23897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:51.735044 23897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:51.775063 23897 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:37281
I20260811 00:13:51.775074 24138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:37281 every 8 connection(s)
I20260811 00:13:51.777042 23897 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:13:51.782171 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 23897
I20260811 00:13:51.782341 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:13:51.782886 7544 external_mini_cluster.cc:1428] Setting key daaa38a4c67d34a9ec512f9b696e375d
I20260811 00:13:51.786302 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:34001
--builtin_ntp_servers=127.7.94.20:45383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 00:13:51.794062 23895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:51.780135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:44245 (local address 127.7.94.62:34001)
0811 00:13:51.780315 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:51.780319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:51.782557 (+ 2238us) server_negotiation.cc:407] Connection header received
0811 00:13:51.784797 (+ 2240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:51.784802 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:51.784887 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:51.784982 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:51.786500 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:51.787596 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:51.790243 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:51.791058 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:51.791212 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:51.793012 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:51.793021 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:51.793028 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:13:51.793054 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:51.793103 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:13:51.793125 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:51.793249 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:51.793554 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 00:13:51.793676 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:13:51.795333 24143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:51.781054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:34001 (local address 127.7.94.2:44245)
0811 00:13:51.782373 (+ 1319us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:51.782432 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:51.784532 (+ 2100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:51.785359 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:51.785370 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:51.785481 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:51.786286 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:51.786315 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:51.787826 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:51.787834 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:51.790071 (+ 2237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:51.790089 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:51.791304 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:51.791313 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:51.791632 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:51.792579 (+ 947us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:13:51.792585 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:51.792643 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:13:51.792760 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:51.793270 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:51.793284 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:51.793295 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 00:13:51.793493 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 00:13:51.793908 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":387,"threads_started":1}
I20260811 00:13:51.796914 24140 heartbeater.cc:347] Connected to a master server at 127.7.94.62:34001
I20260811 00:13:51.797315 24140 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:51.798393 24140 heartbeater.cc:511] Master 127.7.94.62:34001 requested a full tablet report, sending...
I20260811 00:13:51.800211 23564 ts_manager.cc:194] Registered new tserver with Master: a4cc7c491f464dfebee1416bd8767bb7 (127.7.94.2:37281)
I20260811 00:13:51.801008 23564 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.94.2:44245
W20260811 00:13:52.131748 24145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:52.132027 24145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:52.132198 24145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:52.139703 24145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:52.139825 24145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:52.139995 24145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:13:52.149322 24145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:45383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:34001
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:52.150640 24145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:52.152474 24145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:52.166483 24157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:52.166393 24158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:52.166447 24160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:52.167400 24145 server_base.cc:1034] running on GCE node
I20260811 00:13:52.168069 24145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:52.169025 24145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:52.170256 24145 hybrid_clock.cc:648] HybridClock initialized: now 1786407232170219 us; error 54 us; skew 500 ppm
I20260811 00:13:52.170580 24145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:13:52.172894 24145 webserver.cc:460] Webserver started at http://127.7.94.3:34279/ using document root <none> and password file <none>
I20260811 00:13:52.173700 24145 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:52.173834 24145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:52.174167 24145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:52.177093 24145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "b71ba9d2ad72401f9fe11fe60432f1cf"
format_stamp: "Formatted at 2026-08-11 00:13:52 on dist-test-slave-8hhm"
server_key: "607d79e05bdbe52d3b99db682a0359fe"
server_key_iv: "da97775fca934882020b33e8f937818b"
server_key_version: "encryptionkey@0"
I20260811 00:13:52.178067 24145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "b71ba9d2ad72401f9fe11fe60432f1cf"
format_stamp: "Formatted at 2026-08-11 00:13:52 on dist-test-slave-8hhm"
server_key: "607d79e05bdbe52d3b99db682a0359fe"
server_key_iv: "da97775fca934882020b33e8f937818b"
server_key_version: "encryptionkey@0"
I20260811 00:13:52.183270 24145 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:13:52.186589 24170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:52.187948 24145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 00:13:52.188146 24145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "b71ba9d2ad72401f9fe11fe60432f1cf"
format_stamp: "Formatted at 2026-08-11 00:13:52 on dist-test-slave-8hhm"
server_key: "607d79e05bdbe52d3b99db682a0359fe"
server_key_iv: "da97775fca934882020b33e8f937818b"
server_key_version: "encryptionkey@0"
I20260811 00:13:52.188392 24145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:52.210884 24145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:52.211762 24145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:52.212049 24145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:52.212994 24145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:52.214564 24145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:52.214689 24145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:52.214843 24145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:52.214931 24145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:52.248178 24145 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:45755
I20260811 00:13:52.248274 24375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:45755 every 8 connection(s)
I20260811 00:13:52.249536 24145 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:13:52.258814 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 24145
I20260811 00:13:52.258994 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:13:52.259562 7544 external_mini_cluster.cc:1428] Setting key 4a5753ca71f1cf0711b3f142002973d4
I20260811 00:13:52.262562 23895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:52.252088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:43991 (local address 127.7.94.62:34001)
0811 00:13:52.252310 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:52.252316 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:52.253807 (+ 1491us) server_negotiation.cc:407] Connection header received
0811 00:13:52.255687 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:52.255691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:52.255760 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:52.255846 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:52.256825 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:52.257611 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:52.259357 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:52.259978 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:52.260121 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:52.261676 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:52.261682 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:52.261686 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:13:52.261708 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:52.261753 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:13:52.261773 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:52.261852 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:52.262121 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 00:13:52.262223 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 00:13:52.263486 24380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:52.252663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:34001 (local address 127.7.94.3:43991)
0811 00:13:52.253656 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:52.253691 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:52.255439 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:52.256038 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:52.256046 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:52.256142 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:52.256681 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:52.256697 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:52.257761 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:52.257783 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:52.259192 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:52.259204 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:52.260236 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:52.260243 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:52.260493 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:52.261279 (+ 786us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:13:52.261283 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:52.261343 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:13:52.261452 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:52.261885 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:52.261897 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:52.261904 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 00:13:52.262076 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 00:13:52.262357 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":295,"threads_started":1}
I20260811 00:13:52.264776 24377 heartbeater.cc:347] Connected to a master server at 127.7.94.62:34001
I20260811 00:13:52.265033 24377 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:52.265841 24377 heartbeater.cc:511] Master 127.7.94.62:34001 requested a full tablet report, sending...
I20260811 00:13:52.267164 23564 ts_manager.cc:194] Registered new tserver with Master: b71ba9d2ad72401f9fe11fe60432f1cf (127.7.94.3:45755)
I20260811 00:13:52.267961 23564 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.94.3:43991
I20260811 00:13:52.275305 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 00:13:52.281546 23895 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:52.279857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47572 (local address 127.7.94.62:34001)
0811 00:13:52.280092 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:52.280100 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:52.280455 (+ 355us) server_negotiation.cc:407] Connection header received
0811 00:13:52.280645 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:52.280659 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:52.280750 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:52.280856 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:52.281349 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47572: BlockingRecv error: recv got EOF from 127.0.0.1:47572 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 00:13:52.282343 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 23655
I20260811 00:13:52.290576 24398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:52.283368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60856 (local address 127.7.94.1:35441)
0811 00:13:52.284020 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:52.284027 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:52.284054 (+ 27us) server_negotiation.cc:407] Connection header received
0811 00:13:52.284120 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:52.284126 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:52.284342 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:52.284487 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:52.284838 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:52.285911 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:52.287103 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:52.287739 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:52.287892 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:52.288996 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:52.289004 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:52.289017 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:13:52.289664 (+ 647us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:52.289710 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:13:52.289730 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:52.289863 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:52.290016 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 00:13:52.290170 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":339,"threads_started":1}
I20260811 00:13:52.291980 23879 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:52.371888 23891 heartbeater.cc:503] Master 127.7.94.62:34001 was elected leader, sending a full tablet report...
I20260811 00:13:52.373016 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 23655
W20260811 00:13:52.398572 23541 connection.cc:535] server connection from 127.7.94.1:59519 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 00:13:52.398945 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 23897
I20260811 00:13:52.407212 24406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:52.399914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46202 (local address 127.7.94.2:37281)
0811 00:13:52.400430 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:52.400437 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:52.400465 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:13:52.400532 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:52.400538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:52.400762 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:52.400944 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:52.401428 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:52.402439 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:52.403741 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:52.404317 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:52.404455 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:52.405617 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:52.405625 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:52.405639 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:13:52.406314 (+ 675us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:52.406361 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:13:52.406395 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:52.406539 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:52.406692 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 00:13:52.406834 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":256,"threads_started":1}
I20260811 00:13:52.408506 24123 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:52.489490 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 23897
I20260811 00:13:52.514824 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 24145
I20260811 00:13:52.523296 24409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:52.515859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37258 (local address 127.7.94.3:45755)
0811 00:13:52.516527 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:52.516534 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:52.516566 (+ 32us) server_negotiation.cc:407] Connection header received
0811 00:13:52.516641 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:52.516647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:52.516875 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:52.517053 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:52.517577 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:52.518640 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:52.519830 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:52.520373 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:52.520509 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:52.521832 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:52.521841 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:52.521854 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:13:52.522440 (+ 586us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:52.522486 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:13:52.522505 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:52.522644 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:52.522812 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 00:13:52.522942 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":356,"threads_started":1}
I20260811 00:13:52.524530 24368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:52.605804 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 24145
I20260811 00:13:52.631955 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 23508
I20260811 00:13:52.633179 23608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:52.735858 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 23508
2026-08-11T00:13:52Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2522 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T00:13:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:13:52Z Disabled control of system clock
I20260811 00:13:52.822595 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:44251
--webserver_interface=127.7.94.62
--webserver_port=0
--builtin_ntp_servers=127.7.94.20:33729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:44251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 00:13:53.159198 24422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:53.159554 24422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:53.159734 24422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:53.167066 24422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:13:53.167217 24422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:53.167310 24422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:53.167443 24422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:13:53.167518 24422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:13:53.176515 24422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:33729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:44251
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:44251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:53.177920 24422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:53.179735 24422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:53.191712 24437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:53.191802 24432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:53.191828 24433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:53.192785 24422 server_base.cc:1034] running on GCE node
I20260811 00:13:53.193573 24422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:53.194952 24422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:53.196188 24422 hybrid_clock.cc:648] HybridClock initialized: now 1786407233196110 us; error 72 us; skew 500 ppm
I20260811 00:13:53.196491 24422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:13:53.198706 24422 webserver.cc:460] Webserver started at http://127.7.94.62:39283/ using document root <none> and password file <none>
I20260811 00:13:53.199360 24422 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:53.199446 24422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:53.199710 24422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:53.204484 24422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "6368a06ed84145aeaa2144ffcb2cd94b"
format_stamp: "Formatted at 2026-08-11 00:13:53 on dist-test-slave-8hhm"
server_key: "982dfc884fd8e019465918b2e0fe1937"
server_key_iv: "c8074b3ffafd5c162e3357366f7391fe"
server_key_version: "encryptionkey@0"
I20260811 00:13:53.205317 24422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "6368a06ed84145aeaa2144ffcb2cd94b"
format_stamp: "Formatted at 2026-08-11 00:13:53 on dist-test-slave-8hhm"
server_key: "982dfc884fd8e019465918b2e0fe1937"
server_key_iv: "c8074b3ffafd5c162e3357366f7391fe"
server_key_version: "encryptionkey@0"
I20260811 00:13:53.210637 24422 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 00:13:53.213900 24448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:53.215207 24422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:13:53.215386 24422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "6368a06ed84145aeaa2144ffcb2cd94b"
format_stamp: "Formatted at 2026-08-11 00:13:53 on dist-test-slave-8hhm"
server_key: "982dfc884fd8e019465918b2e0fe1937"
server_key_iv: "c8074b3ffafd5c162e3357366f7391fe"
server_key_version: "encryptionkey@0"
I20260811 00:13:53.215667 24422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:53.221326 24422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:53.222097 24422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:53.222426 24422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:53.240535 24422 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:44251
I20260811 00:13:53.240568 24550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:44251 every 8 connection(s)
I20260811 00:13:53.241899 24422 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:13:53.245582 24552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:53.246301 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 24422
I20260811 00:13:53.246511 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:13:53.247145 7544 external_mini_cluster.cc:1428] Setting key b207d6a265f2ca336c733298cad4331d
I20260811 00:13:53.255264 24560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:53.249141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48624 (local address 127.7.94.62:44251)
0811 00:13:53.250163 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:53.250173 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:53.250219 (+ 46us) server_negotiation.cc:407] Connection header received
0811 00:13:53.251713 (+ 1494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:53.251735 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:53.252181 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:53.252514 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:53.253691 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:53.253696 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:53.253708 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:13:53.253722 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:53.253786 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:13:53.253801 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:53.253965 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:53.254108 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 00:13:53.254372 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":361,"threads_started":1}
I20260811 00:13:53.258358 24552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b: Bootstrap starting.
I20260811 00:13:53.261288 24552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:53.262312 24552 log.cc:824] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b: Log is configured to *not* fsync() on all Append() calls
I20260811 00:13:53.264950 24552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b: No bootstrap required, opened a new log
I20260811 00:13:53.271936 24552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6368a06ed84145aeaa2144ffcb2cd94b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44251 } }
I20260811 00:13:53.272322 24552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:53.272430 24552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6368a06ed84145aeaa2144ffcb2cd94b, State: Initialized, Role: FOLLOWER
I20260811 00:13:53.273046 24552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6368a06ed84145aeaa2144ffcb2cd94b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44251 } }
I20260811 00:13:53.273317 24552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:13:53.273460 24552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:13:53.273633 24552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:53.275736 24552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6368a06ed84145aeaa2144ffcb2cd94b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44251 } }
I20260811 00:13:53.276317 24552 leader_election.cc:302] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6368a06ed84145aeaa2144ffcb2cd94b; no voters:
I20260811 00:13:53.276935 24552 leader_election.cc:288] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:13:53.277087 24562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:13:53.277833 24562 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b [term 1 LEADER]: Becoming Leader. State: Replica: 6368a06ed84145aeaa2144ffcb2cd94b, State: Running, Role: LEADER
I20260811 00:13:53.278427 24562 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6368a06ed84145aeaa2144ffcb2cd94b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44251 } }
I20260811 00:13:53.279011 24552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:13:53.281275 24565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6368a06ed84145aeaa2144ffcb2cd94b. Latest consensus state: current_term: 1 leader_uuid: "6368a06ed84145aeaa2144ffcb2cd94b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6368a06ed84145aeaa2144ffcb2cd94b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44251 } } }
I20260811 00:13:53.281718 24565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:53.281531 24563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6368a06ed84145aeaa2144ffcb2cd94b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6368a06ed84145aeaa2144ffcb2cd94b" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44251 } } }
I20260811 00:13:53.281875 24563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:53.282709 24577 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:13:53.288782 24577 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:13:53.299993 24577 catalog_manager.cc:1347] Generated new cluster ID: 8f73e3c9efb64789befb26dec48c2eec
I20260811 00:13:53.300123 24577 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:13:53.317003 24577 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:13:53.317471 24577 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:13:53.324704 24577 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6368a06ed84145aeaa2144ffcb2cd94b: Generated new TSK 0
I20260811 00:13:53.325529 24577 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:13:53.335326 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:44251
--builtin_ntp_servers=127.7.94.20:33729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 00:13:53.683049 24595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:53.683326 24595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:53.683475 24595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:53.690594 24595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:53.690690 24595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:53.690855 24595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:13:53.699968 24595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:33729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:44251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:53.701288 24595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:53.703104 24595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:53.716264 24602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:53.716209 24607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:53.716297 24604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:53.717336 24595 server_base.cc:1034] running on GCE node
I20260811 00:13:53.717947 24595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:53.718798 24595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:53.720086 24595 hybrid_clock.cc:648] HybridClock initialized: now 1786407233719991 us; error 44 us; skew 500 ppm
I20260811 00:13:53.720428 24595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:13:53.722769 24595 webserver.cc:460] Webserver started at http://127.7.94.1:33141/ using document root <none> and password file <none>
I20260811 00:13:53.723414 24595 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:53.723503 24595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:53.723793 24595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:53.726667 24595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "977f484fcab84e08b1cc8319c25eb811"
format_stamp: "Formatted at 2026-08-11 00:13:53 on dist-test-slave-8hhm"
server_key: "1e3eaa628758c4809ce14d7ce02ad25b"
server_key_iv: "b940e8fba23be05dde03d52e6ca74180"
server_key_version: "encryptionkey@0"
I20260811 00:13:53.727639 24595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "977f484fcab84e08b1cc8319c25eb811"
format_stamp: "Formatted at 2026-08-11 00:13:53 on dist-test-slave-8hhm"
server_key: "1e3eaa628758c4809ce14d7ce02ad25b"
server_key_iv: "b940e8fba23be05dde03d52e6ca74180"
server_key_version: "encryptionkey@0"
I20260811 00:13:53.732688 24595 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:13:53.736012 24623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:53.737371 24595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 00:13:53.737547 24595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "977f484fcab84e08b1cc8319c25eb811"
format_stamp: "Formatted at 2026-08-11 00:13:53 on dist-test-slave-8hhm"
server_key: "1e3eaa628758c4809ce14d7ce02ad25b"
server_key_iv: "b940e8fba23be05dde03d52e6ca74180"
server_key_version: "encryptionkey@0"
I20260811 00:13:53.737775 24595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:53.743597 24595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:53.744390 24595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:53.744657 24595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:53.745616 24595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:53.746987 24595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:53.747119 24595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:53.747241 24595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:53.747327 24595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:53.781682 24595 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:38687
I20260811 00:13:53.781737 24874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:38687 every 8 connection(s)
I20260811 00:13:53.783159 24595 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:13:53.788297 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 24595
I20260811 00:13:53.788518 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:13:53.789170 7544 external_mini_cluster.cc:1428] Setting key 34148048ad72eeaab6cb6756ca00f871
I20260811 00:13:53.792044 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:44251
--builtin_ntp_servers=127.7.94.20:33729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 00:13:53.792677 24879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:53.785874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:56445 (local address 127.7.94.62:44251)
0811 00:13:53.786385 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:53.786390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:53.787454 (+ 1064us) server_negotiation.cc:407] Connection header received
0811 00:13:53.789555 (+ 2101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:53.789562 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:53.789645 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:53.789780 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:53.791293 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:53.791299 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:53.791304 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:13:53.791331 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:53.791375 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:13:53.791387 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:53.791521 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:53.792275 (+ 754us) server_negotiation.cc:299] Negotiation successful
0811 00:13:53.792353 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":288,"threads_started":1}
I20260811 00:13:53.793432 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:53.786316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:44251 (local address 127.7.94.1:56445)
0811 00:13:53.787267 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:53.787313 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:53.789165 (+ 1852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:53.790052 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:53.790060 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:53.790185 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:53.790956 (+ 771us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:13:53.790968 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:53.791010 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:13:53.791102 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:53.791497 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:53.791510 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:53.791518 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 00:13:53.791832 (+ 314us) client_negotiation.cc:240] Negotiation successful
0811 00:13:53.792094 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":285,"threads_started":1}
I20260811 00:13:53.795156 24875 heartbeater.cc:347] Connected to a master server at 127.7.94.62:44251
I20260811 00:13:53.795483 24875 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:53.796135 24875 heartbeater.cc:511] Master 127.7.94.62:44251 requested a full tablet report, sending...
I20260811 00:13:53.798388 24472 ts_manager.cc:194] Registered new tserver with Master: 977f484fcab84e08b1cc8319c25eb811 (127.7.94.1:38687)
W20260811 00:13:54.127856 24882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:54.128288 24882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:54.128443 24882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:54.135898 24882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:54.136003 24882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:54.136113 24882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:13:54.145380 24882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:33729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:44251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:54.146660 24882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:54.148435 24882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:54.162678 24890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:54.162588 24897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:54.162631 24891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:54.163378 24882 server_base.cc:1034] running on GCE node
I20260811 00:13:54.164126 24882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:54.165131 24882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:54.166383 24882 hybrid_clock.cc:648] HybridClock initialized: now 1786407234166322 us; error 53 us; skew 500 ppm
I20260811 00:13:54.166746 24882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:13:54.173491 24882 webserver.cc:460] Webserver started at http://127.7.94.2:37407/ using document root <none> and password file <none>
I20260811 00:13:54.174212 24882 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:54.174340 24882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:54.174671 24882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:54.177498 24882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "468fe23a73a84f50aea0002a09d61223"
format_stamp: "Formatted at 2026-08-11 00:13:54 on dist-test-slave-8hhm"
server_key: "f5f5ed718c21addd55dbed8a336a056a"
server_key_iv: "2528b5857c576323ee8e3cf6601d2f85"
server_key_version: "encryptionkey@0"
I20260811 00:13:54.178249 24882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "468fe23a73a84f50aea0002a09d61223"
format_stamp: "Formatted at 2026-08-11 00:13:54 on dist-test-slave-8hhm"
server_key: "f5f5ed718c21addd55dbed8a336a056a"
server_key_iv: "2528b5857c576323ee8e3cf6601d2f85"
server_key_version: "encryptionkey@0"
I20260811 00:13:54.183226 24882 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:13:54.186581 24911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:54.187961 24882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 00:13:54.188115 24882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "468fe23a73a84f50aea0002a09d61223"
format_stamp: "Formatted at 2026-08-11 00:13:54 on dist-test-slave-8hhm"
server_key: "f5f5ed718c21addd55dbed8a336a056a"
server_key_iv: "2528b5857c576323ee8e3cf6601d2f85"
server_key_version: "encryptionkey@0"
I20260811 00:13:54.188356 24882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:54.194007 24882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:54.194798 24882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:54.195058 24882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:54.196036 24882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:54.197465 24882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:54.197567 24882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:54.197710 24882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:54.197803 24882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:54.232199 24882 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:43747
I20260811 00:13:54.232293 25141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:43747 every 8 connection(s)
I20260811 00:13:54.233558 24882 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:13:54.241205 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 24882
I20260811 00:13:54.241456 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:13:54.242003 7544 external_mini_cluster.cc:1428] Setting key dfdfc75ba60b87f77ff1c7a019402f40
I20260811 00:13:54.242031 24879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:54.236264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:37283 (local address 127.7.94.62:44251)
0811 00:13:54.236451 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:54.236456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:54.237839 (+ 1383us) server_negotiation.cc:407] Connection header received
0811 00:13:54.239437 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:54.239441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:54.239495 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:54.239584 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:54.241079 (+ 1495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:54.241088 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:54.241096 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:13:54.241117 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:54.241161 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:13:54.241177 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:54.241381 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:54.241659 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 00:13:54.241725 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 00:13:54.243184 25149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:54.236734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:44251 (local address 127.7.94.2:37283)
0811 00:13:54.237665 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:54.237702 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:54.239199 (+ 1497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:54.239795 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:54.239802 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:54.239888 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:54.240694 (+ 806us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:13:54.240696 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:54.240733 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:13:54.240864 (+ 131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:54.241350 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:54.241362 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:54.241372 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 00:13:54.241726 (+ 354us) client_negotiation.cc:240] Negotiation successful
0811 00:13:54.241970 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":264,"threads_started":1}
I20260811 00:13:54.244534 25144 heartbeater.cc:347] Connected to a master server at 127.7.94.62:44251
I20260811 00:13:54.244835 25144 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:54.245352 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:44251
--builtin_ntp_servers=127.7.94.20:33729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 00:13:54.245576 25144 heartbeater.cc:511] Master 127.7.94.62:44251 requested a full tablet report, sending...
I20260811 00:13:54.247184 24472 ts_manager.cc:194] Registered new tserver with Master: 468fe23a73a84f50aea0002a09d61223 (127.7.94.2:43747)
W20260811 00:13:54.596720 25151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:54.596968 25151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:54.597143 25151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:54.604161 25151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:54.604254 25151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:54.604382 25151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:13:54.613235 25151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:33729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:44251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:54.614454 25151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:54.616602 25151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:54.630172 25161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:54.630204 25163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:54.630234 25159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:54.631093 25151 server_base.cc:1034] running on GCE node
I20260811 00:13:54.631719 25151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:54.632656 25151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:54.633919 25151 hybrid_clock.cc:648] HybridClock initialized: now 1786407234633856 us; error 49 us; skew 500 ppm
I20260811 00:13:54.634248 25151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:13:54.636394 25151 webserver.cc:460] Webserver started at http://127.7.94.3:36883/ using document root <none> and password file <none>
I20260811 00:13:54.637095 25151 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:54.637275 25151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:54.637645 25151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:54.640391 25151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "375ad9e19ad549a386b7b7f8651239ee"
format_stamp: "Formatted at 2026-08-11 00:13:54 on dist-test-slave-8hhm"
server_key: "979a59277eee17c7c1b7eb994144e78a"
server_key_iv: "9e8cb87966087052e4eeb5d4959024a6"
server_key_version: "encryptionkey@0"
I20260811 00:13:54.641192 25151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "375ad9e19ad549a386b7b7f8651239ee"
format_stamp: "Formatted at 2026-08-11 00:13:54 on dist-test-slave-8hhm"
server_key: "979a59277eee17c7c1b7eb994144e78a"
server_key_iv: "9e8cb87966087052e4eeb5d4959024a6"
server_key_version: "encryptionkey@0"
I20260811 00:13:54.646345 25151 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 00:13:54.649688 25176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:54.650986 25151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 00:13:54.651186 25151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "375ad9e19ad549a386b7b7f8651239ee"
format_stamp: "Formatted at 2026-08-11 00:13:54 on dist-test-slave-8hhm"
server_key: "979a59277eee17c7c1b7eb994144e78a"
server_key_iv: "9e8cb87966087052e4eeb5d4959024a6"
server_key_version: "encryptionkey@0"
I20260811 00:13:54.651409 25151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:54.657084 25151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:54.657920 25151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:54.658257 25151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:54.659135 25151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:54.660463 25151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:54.660584 25151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:54.660727 25151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:54.660820 25151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:54.693847 25151 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:35359
I20260811 00:13:54.693943 25406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:35359 every 8 connection(s)
I20260811 00:13:54.695185 25151 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:13:54.698616 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 25151
I20260811 00:13:54.698810 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:13:54.699245 7544 external_mini_cluster.cc:1428] Setting key bdb0730d54c43dedeb9dc1b36b6ecda0
I20260811 00:13:54.704186 24879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:54.697855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:34905 (local address 127.7.94.62:44251)
0811 00:13:54.698068 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:54.698074 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:54.699566 (+ 1492us) server_negotiation.cc:407] Connection header received
0811 00:13:54.701771 (+ 2205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:54.701776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:54.701851 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:54.701990 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:54.703416 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:54.703422 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:54.703427 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:13:54.703445 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:54.703485 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:13:54.703496 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:54.703621 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:54.703837 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 00:13:54.703904 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 00:13:54.705041 25410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:54.698371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:44251 (local address 127.7.94.3:34905)
0811 00:13:54.699395 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:54.699445 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:54.701445 (+ 2000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:54.702253 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:54.702261 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:54.702351 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:54.703080 (+ 729us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:13:54.703084 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:54.703136 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:13:54.703251 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:54.703589 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:54.703601 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:54.703610 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 00:13:54.703907 (+ 297us) client_negotiation.cc:240] Negotiation successful
0811 00:13:54.704166 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":343,"threads_started":1}
I20260811 00:13:54.706274 25407 heartbeater.cc:347] Connected to a master server at 127.7.94.62:44251
I20260811 00:13:54.706566 25407 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:54.707154 25407 heartbeater.cc:511] Master 127.7.94.62:44251 requested a full tablet report, sending...
I20260811 00:13:54.708489 24472 ts_manager.cc:194] Registered new tserver with Master: 375ad9e19ad549a386b7b7f8651239ee (127.7.94.3:35359)
I20260811 00:13:54.715214 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 00:13:54.721148 24879 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:13:54.719541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48636 (local address 127.7.94.62:44251)
0811 00:13:54.719745 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:54.719753 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:54.720205 (+ 452us) server_negotiation.cc:407] Connection header received
0811 00:13:54.720363 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:54.720366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:54.720408 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:54.720476 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:54.720918 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48636: BlockingRecv error: recv got EOF from 127.0.0.1:48636 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 00:13:54.721513 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 24595
I20260811 00:13:54.724833 25423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:54.722503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58320 (local address 127.7.94.1:38687)
0811 00:13:54.723043 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:54.723049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:54.723074 (+ 25us) server_negotiation.cc:407] Connection header received
0811 00:13:54.723129 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:54.723134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:54.723308 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:54.723426 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:54.724237 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:54.724243 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:54.724253 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:13:54.724271 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:54.724309 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:13:54.724322 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:54.724419 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:54.724496 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 00:13:54.724567 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":276,"threads_started":1}
I20260811 00:13:54.726039 24866 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:54.806696 24875 heartbeater.cc:503] Master 127.7.94.62:44251 was elected leader, sending a full tablet report...
I20260811 00:13:54.808555 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 24595
W20260811 00:13:54.831849 24455 connection.cc:535] server connection from 127.7.94.1:56445 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 00:13:54.832244 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 24882
I20260811 00:13:54.835951 25429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:54.833180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52532 (local address 127.7.94.2:43747)
0811 00:13:54.833731 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:54.833738 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:54.833763 (+ 25us) server_negotiation.cc:407] Connection header received
0811 00:13:54.833831 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:54.833837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:54.834042 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:54.834213 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:54.835224 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:54.835230 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:54.835241 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:13:54.835258 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:54.835294 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:13:54.835307 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:54.835474 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:54.835582 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 00:13:54.835698 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":235,"threads_started":1}
I20260811 00:13:54.837347 25134 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:54.920758 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 24882
I20260811 00:13:54.945382 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 25151
I20260811 00:13:54.949194 25432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:54.946392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36234 (local address 127.7.94.3:35359)
0811 00:13:54.947055 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:54.947062 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:54.947093 (+ 31us) server_negotiation.cc:407] Connection header received
0811 00:13:54.947171 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:54.947177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:54.947392 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:54.947516 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:54.948422 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:54.948429 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:54.948440 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:13:54.948456 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:54.948494 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:13:54.948507 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:54.948641 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:54.948695 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 00:13:54.948809 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":349,"threads_started":1}
I20260811 00:13:54.950512 25401 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:55.032581 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 25151
I20260811 00:13:55.057798 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 24422
I20260811 00:13:55.059060 24532 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:55.162223 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 24422
2026-08-11T00:13:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2429 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:13:55 dist-test-slave-8hhm krb5kdc[25441](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:13:55 dist-test-slave-8hhm krb5kdc[25441](info): set up 2 sockets
Aug 11 00:13:55 dist-test-slave-8hhm krb5kdc[25441](info): commencing operation
krb5kdc: starting...
W20260811 00:13:56.506642 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:13:56 dist-test-slave-8hhm krb5kdc[25441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:13:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:13:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:13:56.773356 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:35113
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:35113
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:57.112674 25462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:57.112946 25462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:57.113046 25462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:57.120003 25462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:13:57.120122 25462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:57.120165 25462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:57.120221 25462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:13:57.120263 25462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:13:57.130142 25462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:35113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:35113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:57.131343 25462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:57.132601 25462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:57.144182 25474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:57.144143 25475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:57.144218 25477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:57.144918 25462 server_base.cc:1034] running on GCE node
I20260811 00:13:57.145685 25462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:57.146967 25462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:57.148172 25462 hybrid_clock.cc:648] HybridClock initialized: now 1786407237148129 us; error 40 us; skew 500 ppm
I20260811 00:13:57.148483 25462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:57 dist-test-slave-8hhm krb5kdc[25441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407237, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:57.151539 25462 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:13:57.152877 25462 webserver.cc:460] Webserver started at http://127.7.94.62:43255/ using document root <none> and password file <none>
I20260811 00:13:57.153575 25462 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:57.153682 25462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:57.153927 25462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:57.156770 25462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "4169e4eda9d847d8845edb4f46aebf44"
format_stamp: "Formatted at 2026-08-11 00:13:57 on dist-test-slave-8hhm"
server_key: "6676ef0e3f842e0eb4c7eff62c8e16d0"
server_key_iv: "a0ec09d22be22175af46d1ba73d5c1e0"
server_key_version: "encryptionkey@0"
I20260811 00:13:57.157533 25462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "4169e4eda9d847d8845edb4f46aebf44"
format_stamp: "Formatted at 2026-08-11 00:13:57 on dist-test-slave-8hhm"
server_key: "6676ef0e3f842e0eb4c7eff62c8e16d0"
server_key_iv: "a0ec09d22be22175af46d1ba73d5c1e0"
server_key_version: "encryptionkey@0"
I20260811 00:13:57.162519 25462 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 00:13:57.165758 25495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:57.166947 25462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 00:13:57.167083 25462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "4169e4eda9d847d8845edb4f46aebf44"
format_stamp: "Formatted at 2026-08-11 00:13:57 on dist-test-slave-8hhm"
server_key: "6676ef0e3f842e0eb4c7eff62c8e16d0"
server_key_iv: "a0ec09d22be22175af46d1ba73d5c1e0"
server_key_version: "encryptionkey@0"
I20260811 00:13:57.167255 25462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:57.196076 25462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:57.200290 25462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:57.200605 25462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:57.218189 25462 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:35113
I20260811 00:13:57.218281 25605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:35113 every 8 connection(s)
I20260811 00:13:57.219554 25462 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:13:57.221110 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 25462
I20260811 00:13:57.221354 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:13:57.221997 7544 external_mini_cluster.cc:1428] Setting key 4c5cc52415ae04249eedc5dc06a43cfa
I20260811 00:13:57.223565 25607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 00:13:57 dist-test-slave-8hhm krb5kdc[25441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:13:57.236724 25607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44: Bootstrap starting.
I20260811 00:13:57.240522 25607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:57.241866 25607 log.cc:824] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44: Log is configured to *not* fsync() on all Append() calls
I20260811 00:13:57.245103 25607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44: No bootstrap required, opened a new log
I20260811 00:13:57.246095 25610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:57.224010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59226 (local address 127.7.94.62:35113)
0811 00:13:57.224831 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:57.224846 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:57.224895 (+ 49us) server_negotiation.cc:407] Connection header received
0811 00:13:57.226832 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:57.226859 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:57.227372 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:57.227799 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:57.229237 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:57.231271 (+ 2034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:57.232526 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:57.233551 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:57.233770 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:57.237610 (+ 3840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:57.237620 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:57.237638 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:57.237697 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:57.241539 (+ 3842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:57.242248 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:57.242259 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:57.242268 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:57.242513 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:57.242964 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:57.242967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:57.242969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:57.243615 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:57.243938 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:57.244376 (+ 438us) server_negotiation.cc:299] Negotiation successful
0811 00:13:57.244785 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":307,"threads_started":1}
I20260811 00:13:57.252996 25607 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4169e4eda9d847d8845edb4f46aebf44" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35113 } }
I20260811 00:13:57.253384 25607 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:57.253468 25607 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4169e4eda9d847d8845edb4f46aebf44, State: Initialized, Role: FOLLOWER
I20260811 00:13:57.253999 25607 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4169e4eda9d847d8845edb4f46aebf44" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35113 } }
I20260811 00:13:57.254205 25607 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:13:57.254338 25607 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:13:57.254509 25607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:57.256810 25607 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4169e4eda9d847d8845edb4f46aebf44" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35113 } }
I20260811 00:13:57.257288 25607 leader_election.cc:302] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4169e4eda9d847d8845edb4f46aebf44; no voters:
I20260811 00:13:57.257846 25607 leader_election.cc:288] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:13:57.257977 25613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:13:57.258690 25613 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44 [term 1 LEADER]: Becoming Leader. State: Replica: 4169e4eda9d847d8845edb4f46aebf44, State: Running, Role: LEADER
I20260811 00:13:57.259331 25613 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4169e4eda9d847d8845edb4f46aebf44" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35113 } }
I20260811 00:13:57.260062 25607 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:13:57.262470 25619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4169e4eda9d847d8845edb4f46aebf44. Latest consensus state: current_term: 1 leader_uuid: "4169e4eda9d847d8845edb4f46aebf44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4169e4eda9d847d8845edb4f46aebf44" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35113 } } }
I20260811 00:13:57.262897 25619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44 [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:57.262712 25617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4169e4eda9d847d8845edb4f46aebf44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4169e4eda9d847d8845edb4f46aebf44" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 35113 } } }
I20260811 00:13:57.263118 25617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44 [sys.catalog]: This master's current role is: LEADER
I20260811 00:13:57.264700 25632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:13:57.268864 25632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:13:57.277107 25632 catalog_manager.cc:1347] Generated new cluster ID: 9f3b1812ea2a4818a68ca284a7cda36a
I20260811 00:13:57.277204 25632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:13:57.302593 25632 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:13:57.304514 25632 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:13:57.312305 25632 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44: Generated new TSK 0
I20260811 00:13:57.313263 25632 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
I20260811 00:13:57.385403 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:35113
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:57.720250 25659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:57.720497 25659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:57.720588 25659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:57.727535 25659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:57.727630 25659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:57.727735 25659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:13:57.736701 25659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:35113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:57.737887 25659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:57.739142 25659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:57.752190 25668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:57.752175 25676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:57.752234 25671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:57.753093 25659 server_base.cc:1034] running on GCE node
I20260811 00:13:57.753741 25659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:57.754652 25659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:57.755892 25659 hybrid_clock.cc:648] HybridClock initialized: now 1786407237755849 us; error 42 us; skew 500 ppm
I20260811 00:13:57.756227 25659 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:57 dist-test-slave-8hhm krb5kdc[25441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407237, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:57.759529 25659 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:13:57.761200 25659 webserver.cc:460] Webserver started at http://127.7.94.1:34777/ using document root <none> and password file <none>
I20260811 00:13:57.761993 25659 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:57.762164 25659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:57.762486 25659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:57.765702 25659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "0463933a7d1a4d0a8d692f0b86e8abd7"
format_stamp: "Formatted at 2026-08-11 00:13:57 on dist-test-slave-8hhm"
server_key: "aebb22a5503c04834c81741bd47cb556"
server_key_iv: "09497635349085dbc208b9c2a84e3ef8"
server_key_version: "encryptionkey@0"
I20260811 00:13:57.766549 25659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "0463933a7d1a4d0a8d692f0b86e8abd7"
format_stamp: "Formatted at 2026-08-11 00:13:57 on dist-test-slave-8hhm"
server_key: "aebb22a5503c04834c81741bd47cb556"
server_key_iv: "09497635349085dbc208b9c2a84e3ef8"
server_key_version: "encryptionkey@0"
I20260811 00:13:57.772116 25659 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 00:13:57.775511 25687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:57.776787 25659 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 00:13:57.776949 25659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "0463933a7d1a4d0a8d692f0b86e8abd7"
format_stamp: "Formatted at 2026-08-11 00:13:57 on dist-test-slave-8hhm"
server_key: "aebb22a5503c04834c81741bd47cb556"
server_key_iv: "09497635349085dbc208b9c2a84e3ef8"
server_key_version: "encryptionkey@0"
I20260811 00:13:57.777171 25659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:57.790489 25659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:57.794258 25659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:57.794557 25659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:57.795492 25659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:57.796803 25659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:57.796908 25659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:57.797056 25659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:57.797133 25659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:57.830808 25659 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:38039
I20260811 00:13:57.830857 25897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:38039 every 8 connection(s)
I20260811 00:13:57.832091 25659 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:13:57.842293 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 25659
I20260811 00:13:57.842510 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:13:57.843199 7544 external_mini_cluster.cc:1428] Setting key 8491088f7a162ea966ab5e31fe569f7c
Aug 11 00:13:57 dist-test-slave-8hhm krb5kdc[25441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407237, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:13:57.851912 25904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:57.834714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:39989 (local address 127.7.94.62:35113)
0811 00:13:57.835164 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:57.835171 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:57.836188 (+ 1017us) server_negotiation.cc:407] Connection header received
0811 00:13:57.838059 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:57.838064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:57.838148 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:57.838262 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:57.839653 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:57.840386 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:57.841927 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:57.842299 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:57.842458 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:57.846418 (+ 3960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:57.846423 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:57.846426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:57.846457 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:57.848816 (+ 2359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:57.849705 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:57.849709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:57.849711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:57.849826 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:57.850293 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:57.850296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:57.850297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:57.850523 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:57.850644 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:57.851456 (+ 812us) server_negotiation.cc:299] Negotiation successful
0811 00:13:57.851630 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":213,"threads_started":1}
I20260811 00:13:57.853364 25905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:57.835303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35113 (local address 127.7.94.1:39989)
0811 00:13:57.836010 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:57.836071 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:57.837771 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:57.838470 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:57.838477 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:57.838954 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:57.839482 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:57.839499 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:57.840539 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:57.840543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:57.841789 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:57.841801 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:57.842569 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:57.842577 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:57.842807 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:57.843490 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:57.843512 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:57.846221 (+ 2709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:57.848955 (+ 2734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:57.848965 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:57.848981 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:57.849594 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:57.849925 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:57.849931 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:57.849937 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:57.850176 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:57.850637 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:57.850646 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:57.851079 (+ 433us) client_negotiation.cc:769] Sending connection context
0811 00:13:57.851370 (+ 291us) client_negotiation.cc:240] Negotiation successful
0811 00:13:57.851955 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":289,"threads_started":1}
I20260811 00:13:57.854835 25899 heartbeater.cc:347] Connected to a master server at 127.7.94.62:35113
I20260811 00:13:57.855145 25899 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:57.855981 25899 heartbeater.cc:511] Master 127.7.94.62:35113 requested a full tablet report, sending...
I20260811 00:13:57.858054 25537 ts_manager.cc:194] Registered new tserver with Master: 0463933a7d1a4d0a8d692f0b86e8abd7 (127.7.94.1:38039)
I20260811 00:13:57.859331 25537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:39989
WARNING: no policy specified for kudu/127.7.94.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
I20260811 00:13:57.902479 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:35113
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:58.239068 25909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:58.239357 25909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:58.239478 25909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:58.246585 25909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:58.246688 25909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:58.246868 25909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:13:58.256299 25909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:35113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:58.257736 25909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:58.259104 25909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:58.272548 25923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:58.272800 25922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:58.272891 25926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:58.273605 25909 server_base.cc:1034] running on GCE node
I20260811 00:13:58.274135 25909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:58.275014 25909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:58.276208 25909 hybrid_clock.cc:648] HybridClock initialized: now 1786407238276152 us; error 53 us; skew 500 ppm
I20260811 00:13:58.276490 25909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:58 dist-test-slave-8hhm krb5kdc[25441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407238, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:58.279546 25909 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:13:58.280969 25909 webserver.cc:460] Webserver started at http://127.7.94.2:33003/ using document root <none> and password file <none>
I20260811 00:13:58.281733 25909 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:58.281884 25909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:58.282193 25909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:58.284933 25909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "084d91630c0a44c5a9c13a3fbecdad5c"
format_stamp: "Formatted at 2026-08-11 00:13:58 on dist-test-slave-8hhm"
server_key: "62a502b4d33c1509add6ed70df0064d4"
server_key_iv: "2c235537c8db6756086717f698040558"
server_key_version: "encryptionkey@0"
I20260811 00:13:58.285691 25909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "084d91630c0a44c5a9c13a3fbecdad5c"
format_stamp: "Formatted at 2026-08-11 00:13:58 on dist-test-slave-8hhm"
server_key: "62a502b4d33c1509add6ed70df0064d4"
server_key_iv: "2c235537c8db6756086717f698040558"
server_key_version: "encryptionkey@0"
I20260811 00:13:58.290378 25909 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 00:13:58.293675 25943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:58.294879 25909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 00:13:58.295046 25909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "084d91630c0a44c5a9c13a3fbecdad5c"
format_stamp: "Formatted at 2026-08-11 00:13:58 on dist-test-slave-8hhm"
server_key: "62a502b4d33c1509add6ed70df0064d4"
server_key_iv: "2c235537c8db6756086717f698040558"
server_key_version: "encryptionkey@0"
I20260811 00:13:58.295241 25909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:58.315117 25909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:58.318918 25909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:58.319190 25909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:58.320096 25909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:58.321404 25909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:58.321502 25909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:58.321643 25909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:58.321717 25909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:58.355080 25909 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:34481
I20260811 00:13:58.355132 26147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:34481 every 8 connection(s)
I20260811 00:13:58.356344 25909 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:13:58.366477 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 25909
I20260811 00:13:58.366722 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:13:58.367291 7544 external_mini_cluster.cc:1428] Setting key 488f289ef9163f2387fcc75af52a4efe
Aug 11 00:13:58 dist-test-slave-8hhm krb5kdc[25441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407238, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:13:58.376725 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:58.358791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:58645 (local address 127.7.94.62:35113)
0811 00:13:58.359189 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:58.359193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:58.360191 (+ 998us) server_negotiation.cc:407] Connection header received
0811 00:13:58.362168 (+ 1977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:58.362172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:58.362239 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:58.362326 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:58.363729 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.364535 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.366162 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.366562 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.366698 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:58.370828 (+ 4130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:58.370833 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:58.370836 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:58.370866 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:58.373696 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:58.374571 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:58.374574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:58.374576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:58.374696 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:58.375157 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:58.375159 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:58.375161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:58.375396 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:58.375515 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:58.376306 (+ 791us) server_negotiation.cc:299] Negotiation successful
0811 00:13:58.376473 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":196,"threads_started":1}
I20260811 00:13:58.378211 26156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:58.359305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35113 (local address 127.7.94.2:58645)
0811 00:13:58.360036 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:58.360077 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:58.361879 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:58.362539 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:58.362546 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:58.363054 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:58.363577 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:58.363594 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.364663 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.364666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:58.366039 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:58.366051 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.366808 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.366817 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:58.367104 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:58.367815 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:58.367869 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:58.370608 (+ 2739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:58.373827 (+ 3219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:58.373839 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:58.373857 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:58.374456 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:58.374794 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:58.374802 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:58.374808 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:58.375053 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:58.375515 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:58.375525 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:58.375948 (+ 423us) client_negotiation.cc:769] Sending connection context
0811 00:13:58.376233 (+ 285us) client_negotiation.cc:240] Negotiation successful
0811 00:13:58.376802 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":263,"threads_started":1}
I20260811 00:13:58.379539 26149 heartbeater.cc:347] Connected to a master server at 127.7.94.62:35113
I20260811 00:13:58.379856 26149 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:58.380534 26149 heartbeater.cc:511] Master 127.7.94.62:35113 requested a full tablet report, sending...
I20260811 00:13:58.382035 25537 ts_manager.cc:194] Registered new tserver with Master: 084d91630c0a44c5a9c13a3fbecdad5c (127.7.94.2:34481)
I20260811 00:13:58.383250 25537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:58645
WARNING: no policy specified for kudu/127.7.94.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
I20260811 00:13:58.427783 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:35113
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:13:58.760141 26161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:13:58.760414 26161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:13:58.760599 26161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:13:58.767771 26161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:13:58.767896 26161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:13:58.768088 26161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:13:58.777755 26161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:35113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:13:58.779039 26161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:13:58.780413 26161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:13:58.793398 26178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:58.793431 26180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:13:58.793536 26174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:13:58.794535 26161 server_base.cc:1034] running on GCE node
I20260811 00:13:58.795408 26161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:13:58.796320 26161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:13:58.797506 26161 hybrid_clock.cc:648] HybridClock initialized: now 1786407238797460 us; error 30 us; skew 500 ppm
I20260811 00:13:58.797813 26161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:13:58 dist-test-slave-8hhm krb5kdc[25441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407238, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:13:58.801095 26161 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:13:58.802613 26161 webserver.cc:460] Webserver started at http://127.7.94.3:46615/ using document root <none> and password file <none>
I20260811 00:13:58.803324 26161 fs_manager.cc:359] Metadata directory not provided
I20260811 00:13:58.803468 26161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:13:58.803773 26161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:13:58.806663 26161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "1270f3fff83e42a087bf1fca8cd3300a"
format_stamp: "Formatted at 2026-08-11 00:13:58 on dist-test-slave-8hhm"
server_key: "c41c63731bba9eba855a90a5dc78bd76"
server_key_iv: "f7d06e6ae46f4d95b45356af42cb2189"
server_key_version: "encryptionkey@0"
I20260811 00:13:58.807454 26161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "1270f3fff83e42a087bf1fca8cd3300a"
format_stamp: "Formatted at 2026-08-11 00:13:58 on dist-test-slave-8hhm"
server_key: "c41c63731bba9eba855a90a5dc78bd76"
server_key_iv: "f7d06e6ae46f4d95b45356af42cb2189"
server_key_version: "encryptionkey@0"
I20260811 00:13:58.812601 26161 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 00:13:58.816040 26195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:58.817441 26161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 00:13:58.817605 26161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "1270f3fff83e42a087bf1fca8cd3300a"
format_stamp: "Formatted at 2026-08-11 00:13:58 on dist-test-slave-8hhm"
server_key: "c41c63731bba9eba855a90a5dc78bd76"
server_key_iv: "f7d06e6ae46f4d95b45356af42cb2189"
server_key_version: "encryptionkey@0"
I20260811 00:13:58.817798 26161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:13:58.844995 26161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:13:58.848918 26161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:13:58.849261 26161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:13:58.850273 26161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:13:58.851573 26161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:13:58.851672 26161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:58.851821 26161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:13:58.851890 26161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:13:58.862175 25899 heartbeater.cc:503] Master 127.7.94.62:35113 was elected leader, sending a full tablet report...
I20260811 00:13:58.890600 26161 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:44415
I20260811 00:13:58.890657 26406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:44415 every 8 connection(s)
I20260811 00:13:58.892045 26161 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:13:58.893082 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 26161
I20260811 00:13:58.893404 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:13:58.893976 7544 external_mini_cluster.cc:1428] Setting key ee3649593190b490af70ba8ff652975c
Aug 11 00:13:58 dist-test-slave-8hhm krb5kdc[25441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407238, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:13:58.912249 26410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:58.895055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:57907 (local address 127.7.94.62:35113)
0811 00:13:58.895507 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:58.895513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:58.896413 (+ 900us) server_negotiation.cc:407] Connection header received
0811 00:13:58.898318 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:58.898322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:58.898410 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:58.898509 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:58.899975 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.900661 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.902128 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.902505 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.902607 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:58.905994 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:58.906012 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:58.906015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:58.906043 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:58.908749 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:58.909666 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:58.909670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:58.909673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:58.909791 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:58.910204 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:58.910209 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:58.910211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:58.910638 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:58.910904 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:58.911626 (+ 722us) server_negotiation.cc:299] Negotiation successful
0811 00:13:58.911864 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":203,"threads_started":1}
I20260811 00:13:58.913111 26411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:58.895396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:35113 (local address 127.7.94.3:57907)
0811 00:13:58.896241 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:58.896292 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:58.898095 (+ 1803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:58.898703 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:58.898711 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:58.899214 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:13:58.899811 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:58.899829 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.900808 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.900812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:58.902011 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:58.902023 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.902680 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.902702 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:58.902888 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:58.903564 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:13:58.903585 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:13:58.905818 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:13:58.908902 (+ 3084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:58.908910 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:58.908925 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:58.909547 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:58.909880 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:13:58.909883 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:13:58.909886 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:13:58.910050 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:13:58.910879 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:13:58.910887 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:13:58.911236 (+ 349us) client_negotiation.cc:769] Sending connection context
0811 00:13:58.911515 (+ 279us) client_negotiation.cc:240] Negotiation successful
0811 00:13:58.912055 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":313,"threads_started":1}
I20260811 00:13:58.914372 26407 heartbeater.cc:347] Connected to a master server at 127.7.94.62:35113
I20260811 00:13:58.914630 26407 heartbeater.cc:464] Registering TS with master...
I20260811 00:13:58.915239 26407 heartbeater.cc:511] Master 127.7.94.62:35113 requested a full tablet report, sending...
I20260811 00:13:58.916446 25537 ts_manager.cc:194] Registered new tserver with Master: 1270f3fff83e42a087bf1fca8cd3300a (127.7.94.3:44415)
I20260811 00:13:58.917156 25537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:57907
I20260811 00:13:58.921574 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:13:58.937422 26410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:58.926137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59240 (local address 127.7.94.62:35113)
0811 00:13:58.926286 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:58.926291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:58.926770 (+ 479us) server_negotiation.cc:407] Connection header received
0811 00:13:58.926893 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:58.926896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:58.926950 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:58.927025 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:58.928339 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.928988 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.930195 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.930571 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.930672 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:58.932837 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:58.932841 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:58.932844 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:58.932868 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:58.935185 (+ 2317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:58.935791 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:58.935795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:58.935797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:58.935905 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:58.936203 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:58.936206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:58.936207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:58.936489 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:58.936701 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:58.936967 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 00:13:58.937122 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 00:13:58.941267 25537 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:13:58.948172 25537 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:13:58.969862 26430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:58.959434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38039 (local address 127.0.0.1:43802)
0811 00:13:58.960278 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:58.960314 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:58.960482 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:58.962578 (+ 2096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:58.962590 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:58.962645 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:58.963058 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:58.963067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.965411 (+ 2344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.965417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:58.967239 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:58.967255 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.968769 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.968776 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:58.968973 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:58.969026 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 00:13:58.969150 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 00:13:58.969384 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"mutex_wait_us":2,"thread_start_us":1276,"threads_started":1}
I20260811 00:13:58.970687 26428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:58.959595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:44415 (local address 127.0.0.1:57368)
0811 00:13:58.960959 (+ 1364us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:58.960975 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:58.962628 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:58.963304 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:58.963310 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:58.963334 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:58.963720 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:58.963736 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.965650 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.965655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:58.967874 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:58.967888 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.969709 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.969719 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:58.969907 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:58.969937 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 00:13:58.970026 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 00:13:58.970177 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1239,"mutex_wait_us":4,"thread_start_us":383,"threads_started":1}
I20260811 00:13:58.970829 26432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:58.959791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43802 (local address 127.7.94.1:38039)
0811 00:13:58.961766 (+ 1975us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:58.961778 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:58.961827 (+ 49us) server_negotiation.cc:407] Connection header received
0811 00:13:58.961951 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:58.961962 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:58.962259 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:58.963216 (+ 957us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:58.963566 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.965253 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.967393 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.968542 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.969979 (+ 1437us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:58.970197 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:58.970295 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 00:13:58.970487 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1747,"thread_start_us":1572,"threads_started":1}
I20260811 00:13:58.970975 26437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:58.961142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:34481 (local address 127.0.0.1:46106)
0811 00:13:58.962116 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:58.962133 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:58.962247 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:58.963628 (+ 1381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:58.963632 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:58.963660 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:58.964098 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:58.964112 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.966070 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.966075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:58.968188 (+ 2113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:58.968201 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.970127 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.970133 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:58.970341 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:58.970373 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 00:13:58.970459 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 00:13:58.970600 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":852,"thread_start_us":1184,"threads_started":1}
I20260811 00:13:58.971606 26439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:58.961451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46106 (local address 127.7.94.2:34481)
0811 00:13:58.962972 (+ 1521us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:58.962980 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:58.963004 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:13:58.963093 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:58.963099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:58.963311 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:58.963492 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:58.964286 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.965927 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.968373 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.969664 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.969801 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:58.969992 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:58.971213 (+ 1221us) server_negotiation.cc:299] Negotiation successful
0811 00:13:58.971375 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1366,"thread_start_us":1215,"threads_started":1}
I20260811 00:13:58.971612 26434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:58.960235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57368 (local address 127.7.94.3:44415)
0811 00:13:58.961144 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:58.961153 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:58.961187 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:13:58.962869 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:58.962877 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:58.963092 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:58.963297 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:58.963960 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.965532 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.968040 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:58.969469 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:58.969607 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:58.969856 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:58.970717 (+ 861us) server_negotiation.cc:299] Negotiation successful
0811 00:13:58.971301 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718,"thread_start_us":1093,"threads_started":1}
I20260811 00:13:58.974697 25766 tablet_service.cc:1467] Processing CreateTablet for tablet 724f380dbf1b4b94ae61f33902f22515 (DEFAULT_TABLE table=test-table [id=3cb1a49e39de4e81b0d2c6e3ce05e293]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:58.975337 26026 tablet_service.cc:1467] Processing CreateTablet for tablet 724f380dbf1b4b94ae61f33902f22515 (DEFAULT_TABLE table=test-table [id=3cb1a49e39de4e81b0d2c6e3ce05e293]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:58.975343 26283 tablet_service.cc:1467] Processing CreateTablet for tablet 724f380dbf1b4b94ae61f33902f22515 (DEFAULT_TABLE table=test-table [id=3cb1a49e39de4e81b0d2c6e3ce05e293]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:13:58.976046 25766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 724f380dbf1b4b94ae61f33902f22515. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:58.976711 26026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 724f380dbf1b4b94ae61f33902f22515. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:58.976711 26283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 724f380dbf1b4b94ae61f33902f22515. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:13:58.987406 26442 tablet_bootstrap.cc:492] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7: Bootstrap starting.
I20260811 00:13:58.991546 26442 tablet_bootstrap.cc:654] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:58.992601 26442 log.cc:824] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7: Log is configured to *not* fsync() on all Append() calls
I20260811 00:13:58.993562 26446 tablet_bootstrap.cc:492] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a: Bootstrap starting.
I20260811 00:13:58.994529 26443 tablet_bootstrap.cc:492] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c: Bootstrap starting.
I20260811 00:13:58.996647 26442 tablet_bootstrap.cc:492] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7: No bootstrap required, opened a new log
I20260811 00:13:58.996918 26442 ts_tablet_manager.cc:1397] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260811 00:13:58.998168 26446 tablet_bootstrap.cc:654] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:58.998158 26443 tablet_bootstrap.cc:654] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c: Neither blocks nor log segments found. Creating new log.
I20260811 00:13:58.999467 26443 log.cc:824] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c: Log is configured to *not* fsync() on all Append() calls
I20260811 00:13:58.999462 26446 log.cc:824] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a: Log is configured to *not* fsync() on all Append() calls
I20260811 00:13:59.002187 26443 tablet_bootstrap.cc:492] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c: No bootstrap required, opened a new log
I20260811 00:13:59.002434 26443 ts_tablet_manager.cc:1397] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260811 00:13:59.002651 26446 tablet_bootstrap.cc:492] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a: No bootstrap required, opened a new log
I20260811 00:13:59.002910 26446 ts_tablet_manager.cc:1397] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260811 00:13:59.007217 26442 raft_consensus.cc:348] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0463933a7d1a4d0a8d692f0b86e8abd7" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38039 } } peers { permanent_uuid: "1270f3fff83e42a087bf1fca8cd3300a" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44415 } } peers { permanent_uuid: "084d91630c0a44c5a9c13a3fbecdad5c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34481 } }
I20260811 00:13:59.007647 26442 raft_consensus.cc:374] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:59.007746 26442 raft_consensus.cc:729] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0463933a7d1a4d0a8d692f0b86e8abd7, State: Initialized, Role: FOLLOWER
I20260811 00:13:59.008376 26442 consensus_queue.cc:260] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0463933a7d1a4d0a8d692f0b86e8abd7" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38039 } } peers { permanent_uuid: "1270f3fff83e42a087bf1fca8cd3300a" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44415 } } peers { permanent_uuid: "084d91630c0a44c5a9c13a3fbecdad5c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34481 } }
I20260811 00:13:59.009902 26443 raft_consensus.cc:348] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0463933a7d1a4d0a8d692f0b86e8abd7" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38039 } } peers { permanent_uuid: "1270f3fff83e42a087bf1fca8cd3300a" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44415 } } peers { permanent_uuid: "084d91630c0a44c5a9c13a3fbecdad5c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34481 } }
I20260811 00:13:59.010250 26442 ts_tablet_manager.cc:1428] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260811 00:13:59.010326 26443 raft_consensus.cc:374] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:59.010417 26443 raft_consensus.cc:729] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 084d91630c0a44c5a9c13a3fbecdad5c, State: Initialized, Role: FOLLOWER
I20260811 00:13:59.011073 26443 consensus_queue.cc:260] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0463933a7d1a4d0a8d692f0b86e8abd7" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38039 } } peers { permanent_uuid: "1270f3fff83e42a087bf1fca8cd3300a" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44415 } } peers { permanent_uuid: "084d91630c0a44c5a9c13a3fbecdad5c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34481 } }
I20260811 00:13:59.012240 26149 heartbeater.cc:503] Master 127.7.94.62:35113 was elected leader, sending a full tablet report...
I20260811 00:13:59.012880 26443 ts_tablet_manager.cc:1428] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260811 00:13:59.014108 26446 raft_consensus.cc:348] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0463933a7d1a4d0a8d692f0b86e8abd7" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38039 } } peers { permanent_uuid: "1270f3fff83e42a087bf1fca8cd3300a" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44415 } } peers { permanent_uuid: "084d91630c0a44c5a9c13a3fbecdad5c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34481 } }
I20260811 00:13:59.014520 26446 raft_consensus.cc:374] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:13:59.014624 26446 raft_consensus.cc:729] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1270f3fff83e42a087bf1fca8cd3300a, State: Initialized, Role: FOLLOWER
I20260811 00:13:59.015226 26446 consensus_queue.cc:260] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0463933a7d1a4d0a8d692f0b86e8abd7" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38039 } } peers { permanent_uuid: "1270f3fff83e42a087bf1fca8cd3300a" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44415 } } peers { permanent_uuid: "084d91630c0a44c5a9c13a3fbecdad5c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34481 } }
I20260811 00:13:59.016402 26407 heartbeater.cc:503] Master 127.7.94.62:35113 was elected leader, sending a full tablet report...
I20260811 00:13:59.016855 26446 ts_tablet_manager.cc:1428] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260811 00:13:59.057834 26450 raft_consensus.cc:482] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:13:59.058203 26450 raft_consensus.cc:504] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0463933a7d1a4d0a8d692f0b86e8abd7" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38039 } } peers { permanent_uuid: "1270f3fff83e42a087bf1fca8cd3300a" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44415 } } peers { permanent_uuid: "084d91630c0a44c5a9c13a3fbecdad5c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34481 } }
I20260811 00:13:59.060070 26450 leader_election.cc:288] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1270f3fff83e42a087bf1fca8cd3300a (127.7.94.3:44415), 084d91630c0a44c5a9c13a3fbecdad5c (127.7.94.2:34481)
I20260811 00:13:59.067029 26434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:59.060342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:56359 (local address 127.7.94.3:44415)
0811 00:13:59.060593 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:59.060603 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:59.061338 (+ 735us) server_negotiation.cc:407] Connection header received
0811 00:13:59.061548 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:59.061555 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:59.061637 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:59.061736 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:59.062404 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:59.063170 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:59.065174 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:59.066017 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:59.066142 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:59.066278 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:59.066630 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 00:13:59.066763 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 00:13:59.067193 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:59.060269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:44415 (local address 127.7.94.1:56359)
0811 00:13:59.061176 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:59.061200 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:59.061396 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:59.061760 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:59.061765 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:59.061788 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:59.062245 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:59.062257 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:59.063306 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:59.063312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:59.065010 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:59.065024 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:59.066265 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:59.066271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:59.066429 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:59.066483 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 00:13:59.066575 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 00:13:59.066713 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"mutex_wait_us":31,"thread_start_us":523,"threads_started":1}
I20260811 00:13:59.067251 26456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:59.060552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:34481 (local address 127.7.94.1:51281)
0811 00:13:59.061437 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0811 00:13:59.061467 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 00:13:59.061597 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:13:59.061951 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:13:59.061954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:13:59.061974 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:13:59.062380 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:59.062392 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:59.063376 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:59.063380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:59.065044 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:13:59.065058 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:13:59.066307 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:13:59.066318 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:13:59.066490 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:59.066511 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 00:13:59.066642 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 00:13:59.066772 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":769,"mutex_wait_us":64,"thread_start_us":527,"threads_started":1}
I20260811 00:13:59.067049 26439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:59.060506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:51281 (local address 127.7.94.2:34481)
0811 00:13:59.060886 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:59.060892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:59.061560 (+ 668us) server_negotiation.cc:407] Connection header received
0811 00:13:59.061739 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:59.061744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:59.061818 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:59.061923 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:13:59.062534 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:59.063217 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:59.065247 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:59.066024 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:59.066187 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:59.066289 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:59.066681 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 00:13:59.066801 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190}
I20260811 00:13:59.068282 26319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "724f380dbf1b4b94ae61f33902f22515" candidate_uuid: "0463933a7d1a4d0a8d692f0b86e8abd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1270f3fff83e42a087bf1fca8cd3300a" is_pre_election: true
I20260811 00:13:59.068392 26063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "724f380dbf1b4b94ae61f33902f22515" candidate_uuid: "0463933a7d1a4d0a8d692f0b86e8abd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "084d91630c0a44c5a9c13a3fbecdad5c" is_pre_election: true
I20260811 00:13:59.068923 26063 raft_consensus.cc:2454] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0463933a7d1a4d0a8d692f0b86e8abd7 in term 0.
I20260811 00:13:59.068923 26319 raft_consensus.cc:2454] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0463933a7d1a4d0a8d692f0b86e8abd7 in term 0.
I20260811 00:13:59.069687 25689 leader_election.cc:302] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0463933a7d1a4d0a8d692f0b86e8abd7, 1270f3fff83e42a087bf1fca8cd3300a; no voters:
I20260811 00:13:59.070119 26450 raft_consensus.cc:2781] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:13:59.070273 26450 raft_consensus.cc:482] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:13:59.070456 26450 raft_consensus.cc:3037] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:59.072809 26450 raft_consensus.cc:504] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0463933a7d1a4d0a8d692f0b86e8abd7" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38039 } } peers { permanent_uuid: "1270f3fff83e42a087bf1fca8cd3300a" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44415 } } peers { permanent_uuid: "084d91630c0a44c5a9c13a3fbecdad5c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34481 } }
I20260811 00:13:59.073462 26450 leader_election.cc:288] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [CANDIDATE]: Term 1 election: Requested vote from peers 1270f3fff83e42a087bf1fca8cd3300a (127.7.94.3:44415), 084d91630c0a44c5a9c13a3fbecdad5c (127.7.94.2:34481)
I20260811 00:13:59.073892 26319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "724f380dbf1b4b94ae61f33902f22515" candidate_uuid: "0463933a7d1a4d0a8d692f0b86e8abd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1270f3fff83e42a087bf1fca8cd3300a"
I20260811 00:13:59.073954 26063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "724f380dbf1b4b94ae61f33902f22515" candidate_uuid: "0463933a7d1a4d0a8d692f0b86e8abd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "084d91630c0a44c5a9c13a3fbecdad5c"
I20260811 00:13:59.074074 26319 raft_consensus.cc:3037] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:59.074200 26063 raft_consensus.cc:3037] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:13:59.076332 26319 raft_consensus.cc:2454] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0463933a7d1a4d0a8d692f0b86e8abd7 in term 1.
I20260811 00:13:59.076423 26063 raft_consensus.cc:2454] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0463933a7d1a4d0a8d692f0b86e8abd7 in term 1.
I20260811 00:13:59.076737 25689 leader_election.cc:302] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0463933a7d1a4d0a8d692f0b86e8abd7, 1270f3fff83e42a087bf1fca8cd3300a; no voters:
I20260811 00:13:59.077039 26450 raft_consensus.cc:2781] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:13:59.077523 26450 raft_consensus.cc:686] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [term 1 LEADER]: Becoming Leader. State: Replica: 0463933a7d1a4d0a8d692f0b86e8abd7, State: Running, Role: LEADER
I20260811 00:13:59.078073 26450 consensus_queue.cc:237] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0463933a7d1a4d0a8d692f0b86e8abd7" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38039 } } peers { permanent_uuid: "1270f3fff83e42a087bf1fca8cd3300a" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44415 } } peers { permanent_uuid: "084d91630c0a44c5a9c13a3fbecdad5c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34481 } }
I20260811 00:13:59.082705 25537 catalog_manager.cc:5495] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0463933a7d1a4d0a8d692f0b86e8abd7 (127.7.94.1). New cstate: current_term: 1 leader_uuid: "0463933a7d1a4d0a8d692f0b86e8abd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0463933a7d1a4d0a8d692f0b86e8abd7" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38039 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1270f3fff83e42a087bf1fca8cd3300a" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "084d91630c0a44c5a9c13a3fbecdad5c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34481 } health_report { overall_health: UNKNOWN } } }
W20260811 00:13:59.084414 25902 tablet.cc:2369] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:13:59.108449 26151 tablet.cc:2369] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:13:59.124019 26432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:59.118904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43806 (local address 127.7.94.1:38039)
0811 00:13:59.119078 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:59.119083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:59.119195 (+ 112us) server_negotiation.cc:407] Connection header received
0811 00:13:59.119356 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:59.119359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:59.119455 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:59.119578 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:13:59.119892 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:59.120581 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:59.122023 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:59.122447 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:59.122612 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:59.123029 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:13:59.123580 (+ 551us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:13:59.123691 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:59.123762 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 00:13:59.123823 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 00:13:59.131927 26063 raft_consensus.cc:1261] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c [term 1 FOLLOWER]: Refusing update from remote peer 0463933a7d1a4d0a8d692f0b86e8abd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:13:59.131942 26319 raft_consensus.cc:1261] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a [term 1 FOLLOWER]: Refusing update from remote peer 0463933a7d1a4d0a8d692f0b86e8abd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:13:59.132853 26450 consensus_queue.cc:1035] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "084d91630c0a44c5a9c13a3fbecdad5c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 34481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:13:59.133241 26458 consensus_queue.cc:1035] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1270f3fff83e42a087bf1fca8cd3300a" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 00:13:59.143710 26408 tablet.cc:2369] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:13:59.166258 25537 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:13:59.166507 25537 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:13:59.170022 25537 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4169e4eda9d847d8845edb4f46aebf44: Sending DeleteTablet for 3 replicas of tablet 724f380dbf1b4b94ae61f33902f22515
I20260811 00:13:59.171295 26283 tablet_service.cc:1514] Processing DeleteTablet for tablet 724f380dbf1b4b94ae61f33902f22515 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:59 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:57368
I20260811 00:13:59.171276 25766 tablet_service.cc:1514] Processing DeleteTablet for tablet 724f380dbf1b4b94ae61f33902f22515 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:59 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:43802
I20260811 00:13:59.172766 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 25659
I20260811 00:13:59.172780 26026 tablet_service.cc:1514] Processing DeleteTablet for tablet 724f380dbf1b4b94ae61f33902f22515 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:13:59 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:46106
I20260811 00:13:59.177991 26482 tablet_replica.cc:330] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c: stopping tablet replica
I20260811 00:13:59.178459 26482 raft_consensus.cc:2229] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:13:59.179018 26482 raft_consensus.cc:2258] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:59.180028 26480 tablet_replica.cc:330] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a: stopping tablet replica
I20260811 00:13:59.180321 26481 tablet_replica.cc:330] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7: stopping tablet replica
I20260811 00:13:59.180496 26480 raft_consensus.cc:2229] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:13:59.180689 26481 raft_consensus.cc:2229] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:13:59.181031 26480 raft_consensus.cc:2258] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:59.181335 26481 raft_consensus.cc:2258] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:13:59.181838 26482 ts_tablet_manager.cc:1905] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:59.183640 26481 ts_tablet_manager.cc:1905] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:59.189154 26480 ts_tablet_manager.cc:1905] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:13:59.189975 26482 ts_tablet_manager.cc:1918] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:13:59.190172 26482 log.cc:1196] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/724f380dbf1b4b94ae61f33902f22515
I20260811 00:13:59.190800 26482 ts_tablet_manager.cc:1939] T 724f380dbf1b4b94ae61f33902f22515 P 084d91630c0a44c5a9c13a3fbecdad5c: Deleting consensus metadata
I20260811 00:13:59.192677 25498 catalog_manager.cc:4841] TS 084d91630c0a44c5a9c13a3fbecdad5c (127.7.94.2:34481): tablet 724f380dbf1b4b94ae61f33902f22515 (table test-table [id=3cb1a49e39de4e81b0d2c6e3ce05e293]) successfully deleted
I20260811 00:13:59.193051 26481 ts_tablet_manager.cc:1918] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:13:59.193349 26481 log.cc:1196] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/724f380dbf1b4b94ae61f33902f22515
I20260811 00:13:59.193976 26481 ts_tablet_manager.cc:1939] T 724f380dbf1b4b94ae61f33902f22515 P 0463933a7d1a4d0a8d692f0b86e8abd7: Deleting consensus metadata
Aug 11 00:13:59 dist-test-slave-8hhm krb5kdc[25441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:13:59.195839 25498 catalog_manager.cc:4841] TS 0463933a7d1a4d0a8d692f0b86e8abd7 (127.7.94.1:38039): tablet 724f380dbf1b4b94ae61f33902f22515 (table test-table [id=3cb1a49e39de4e81b0d2c6e3ce05e293]) successfully deleted
I20260811 00:13:59.197707 26480 ts_tablet_manager.cc:1918] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:13:59.197885 26480 log.cc:1196] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/724f380dbf1b4b94ae61f33902f22515
I20260811 00:13:59.198433 26480 ts_tablet_manager.cc:1939] T 724f380dbf1b4b94ae61f33902f22515 P 1270f3fff83e42a087bf1fca8cd3300a: Deleting consensus metadata
I20260811 00:13:59.200059 25497 catalog_manager.cc:4841] TS 1270f3fff83e42a087bf1fca8cd3300a (127.7.94.3:44415): tablet 724f380dbf1b4b94ae61f33902f22515 (table test-table [id=3cb1a49e39de4e81b0d2c6e3ce05e293]) successfully deleted
I20260811 00:13:59.203246 26432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:59.174224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43812 (local address 127.7.94.1:38039)
0811 00:13:59.174408 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:59.174415 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:59.177672 (+ 3257us) server_negotiation.cc:407] Connection header received
0811 00:13:59.177958 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:59.177965 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:59.178052 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:59.178136 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:59.181794 (+ 3658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:59.183015 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:59.184903 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:59.185735 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:59.185834 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:59.195925 (+ 10091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:59.195940 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:59.195958 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:59.196010 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:59.200191 (+ 4181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:59.200889 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:59.200899 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:59.200903 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:59.201077 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:59.201598 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:59.201604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:59.201607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:59.202083 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:59.202392 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:59.202567 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 00:13:59.202851 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 00:13:59.204083 25893 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:59.311508 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 25659
I20260811 00:13:59.343145 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 25909
Aug 11 00:13:59 dist-test-slave-8hhm krb5kdc[25441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.2@KRBTEST.COM
I20260811 00:13:59.357905 26439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:59.343981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46116 (local address 127.7.94.2:34481)
0811 00:13:59.344138 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:59.344142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:59.344158 (+ 16us) server_negotiation.cc:407] Connection header received
0811 00:13:59.344236 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:59.344239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:59.344308 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:59.344389 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:59.345643 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:59.346336 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:59.347419 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:59.347793 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:59.347878 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:59.351994 (+ 4116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:59.352007 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:59.352023 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:59.352054 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:59.355072 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:59.355831 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:59.355838 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:59.355840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:59.355979 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:59.356357 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:59.356366 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:59.356369 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:59.356766 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:59.357053 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:59.357400 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 00:13:59.357615 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 00:13:59.358685 26143 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:59.446919 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 25909
I20260811 00:13:59.472451 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 26161
Aug 11 00:13:59 dist-test-slave-8hhm krb5kdc[25441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.3@KRBTEST.COM
I20260811 00:13:59.487233 26434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:13:59.473301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57376 (local address 127.7.94.3:44415)
0811 00:13:59.473465 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 00:13:59.473470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:13:59.473507 (+ 37us) server_negotiation.cc:407] Connection header received
0811 00:13:59.473684 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:13:59.473688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:13:59.473754 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:13:59.473842 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:13:59.475046 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:59.475750 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:59.476875 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:13:59.477320 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:13:59.477434 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:13:59.481614 (+ 4180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:13:59.481630 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:13:59.481646 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:13:59.481681 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:13:59.484442 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:59.485096 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:59.485105 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:59.485107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:59.485302 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:13:59.485660 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:13:59.485665 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:13:59.485668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:13:59.486024 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:13:59.486337 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 00:13:59.486669 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 00:13:59.486896 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 00:13:59.487871 26397 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:59.575368 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 26161
I20260811 00:13:59.600767 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 25462
I20260811 00:13:59.601999 25590 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:13:59.709529 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 25462
2026-08-11T00:13:59Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4551 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:13:59 dist-test-slave-8hhm krb5kdc[26495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:13:59 dist-test-slave-8hhm krb5kdc[26495](info): set up 2 sockets
Aug 11 00:13:59 dist-test-slave-8hhm krb5kdc[26495](info): commencing operation
krb5kdc: starting...
W20260811 00:14:01.055100 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.003s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:14:01 dist-test-slave-8hhm krb5kdc[26495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:14:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:14:01Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.94.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.94.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.94.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/rangeradmin_127.7.94.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.94.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/rangeradmin_127.7.94.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.94.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.94.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.94.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/rangerlookup_127.7.94.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.94.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/rangerlookup_127.7.94.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.94.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.94.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.94.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/HTTP_127.7.94.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.94.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/HTTP_127.7.94.20@KRBTEST.COM.keytab.
I20260811 00:14:01.450311 7544 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task1KEo_S/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/postgres -l logfile start
2026-08-11 00:14:03.876 UTC [26569] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 00:14:03.877 UTC [26569] LOG: listening on IPv4 address "127.7.94.20", port 60675
2026-08-11 00:14:03.880 UTC [26569] LOG: listening on Unix socket "/tmp/.s.PGSQL.60675"
2026-08-11 00:14:03.904 UTC [26575] LOG: database system was shut down at 2026-08-11 00:14:02 UTC
2026-08-11 00:14:03.908 UTC [26569] LOG: database system is ready to accept connections
I20260811 00:14:05.137821 7544 mini_postgres.cc:96] Postgres bound to 60675
2026-08-11 00:14:05.163 UTC [26599] FATAL: database "slave" does not exist
127.7.94.20:60675 - accepting connections
I20260811 00:14:05.164474 7544 mini_ranger.cc:161] Starting Ranger...
I20260811 00:14:05.197724 7544 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 00:14:05.647086 7544 mini_ranger.cc:87] Created ranger Postgres database
I20260811 00:14:05.647209 7544 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ranger-admin
2026-08-11 00:14:06,030 [I] DB FLAVOR :POSTGRES
2026-08-11 00:14:06,031 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 00:14:06,031 [I] Checking connection..
2026-08-11 00:14:06,031 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 00:14:06,316 [I] Checking connection passed.
2026-08-11 00:14:06,317 [I] --------- Verifying version history table ---------
2026-08-11 00:14:06,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 00:14:06,582 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 00:14:06,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 00:14:06,831 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 00:14:06,832 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 00:14:06,832 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 00:14:07.078 UTC [26717] WARNING: there is no transaction in progress
2026-08-11 00:14:07,087 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 00:14:07,088 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 00:14:07,325 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 00:14:07,325 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 00:14:07,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 00:14:07,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 00:14:07,839 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8hhm.c.gcp-upstream.internal','N') ;"
2026-08-11 00:14:08,063 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 00:14:08,297 [I] Table x_portal_user does not exist in database ranger
2026-08-11 00:14:08,298 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 00:14:08,545 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 00:14:08,546 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 00:14:08,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 00:14:09,044 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 00:14:09,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 00:14:09.976 UTC [26914] WARNING: there is no transaction in progress
2026-08-11 00:14:09.992 UTC [26914] WARNING: there is no transaction in progress
2026-08-11 00:14:10.008 UTC [26914] WARNING: there is no transaction in progress
2026-08-11 00:14:10.024 UTC [26914] WARNING: there is no transaction in progress
2026-08-11 00:14:10.038 UTC [26914] WARNING: there is no transaction in progress
2026-08-11 00:14:10.056 UTC [26914] WARNING: there is no transaction in progress
2026-08-11 00:14:10.119 UTC [26914] WARNING: there is no transaction in progress
2026-08-11 00:14:10.125 UTC [26914] WARNING: there is no transaction in progress
2026-08-11 00:14:10.139 UTC [26914] WARNING: there is no transaction in progress
2026-08-11 00:14:10.149 UTC [26914] WARNING: there is no transaction in progress
2026-08-11 00:14:10.159 UTC [26914] WARNING: there is no transaction in progress
2026-08-11 00:14:10.172 UTC [26914] WARNING: there is no transaction in progress
2026-08-11 00:14:10.181 UTC [26914] WARNING: there is no transaction in progress
2026-08-11 00:14:10.187 UTC [26914] WARNING: there is no transaction in progress
2026-08-11 00:14:10.194 UTC [26914] WARNING: there is no transaction in progress
2026-08-11 00:14:10.200 UTC [26914] WARNING: there is no transaction in progress
2026-08-11 00:14:10,783 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 00:14:10,783 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 00:14:11,019 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 00:14:11,019 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 00:14:11,277 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 00:14:11,278 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 00:14:11,544 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 00:14:11,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 00:14:11,809 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 00:14:12,053 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8hhm.c.gcp-upstream.internal';"
2026-08-11 00:14:12,288 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 00:14:12,288 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 00:14:12,540 [I] DB_PATCHES have already been applied
I20260811 00:14:12.546703 7544 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ranger-admin:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task1KEo_S/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 00:14:12.546818 7544 mini_ranger.cc:193] Using host: 127.7.94.20
I20260811 00:14:12.568580 7544 mini_ranger.cc:227] Ranger admin URL: http://127.7.94.20:60517
Aug 11, 2026 12:14:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:14:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:14:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 12:14:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 12:14:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 12:14:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 12:14:13 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 00:14:13.949914 7544 mini_ranger.cc:160] Time spent starting Ranger: real 8.785s user 0.003s sys 0.061s
Aug 11, 2026 12:14:34 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 12:14:34 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 12:14:34 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 12:14:34 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 12:14:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 12:14:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 12:14:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 12:14:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 12:14:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 12:14:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 12:14:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 12:14:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 12:14:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 12:14:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 12:14:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 12:14:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 12:14:34 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 12:14:35 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 00:14:36.360589 7544 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.94.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.94.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.94.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/rangerkms_127.7.94.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.94.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/rangerkms_127.7.94.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.94.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.94.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.94.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/HTTP_127.7.94.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.94.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/HTTP_127.7.94.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 00:14:36 dist-test-slave-8hhm krb5kdc[26495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407276, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 00:14:36.582055 7544 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 00:14:36.607985 7544 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 00:14:37.510403 7544 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 00:14:37.510545 7544 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ranger-kms
I20260811 00:14:37.510697 7544 mini_ranger_kms.cc:227] Using postgres at 127.7.94.20:60675
2026-08-11 00:14:38,115 [I] DB FLAVOR :POSTGRES
2026-08-11 00:14:38,116 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 00:14:38,116 [I] Checking connection
2026-08-11 00:14:38,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 00:14:38,375 [I] connection success
2026-08-11 00:14:38,375 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 00:14:38,376 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 00:14:38,376 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 00:14:38,617 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 00:14:38,617 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 00:14:38,617 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 00:14:38,617 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60675/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 00:14:38,896 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 00:14:39.364156 7544 mini_ranger_kms.cc:310] Created kms service
I20260811 00:14:39.568557 7544 mini_ranger_kms.cc:326] Created kudu user
I20260811 00:14:39.614679 7544 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 00:14:39 dist-test-slave-8hhm krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407276, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
I20260811 00:14:40.055683 7544 mini_ranger_kms.cc:384] Added ranger policy
I20260811 00:14:40.057623 7544 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ranger-kms:/tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 00:14:40.057730 7544 mini_ranger_kms.cc:240] Using host: 127.7.94.20
I20260811 00:14:40.086459 7544 mini_ranger_kms.cc:276] Ranger KMS PID: 27252
I20260811 00:14:40.086720 7544 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.94.20:33293
Aug 11, 2026 12:14:41 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:14:41 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:14:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 12:14:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 12:14:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 12:14:41 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 00:14:42 dist-test-slave-8hhm krb5kdc[26495](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786407282, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.94.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 00:14:42 dist-test-slave-8hhm krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407282, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.94.20@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
I20260811 00:14:42.979112 7544 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.397s user 0.154s sys 0.446s
I20260811 00:14:42.979504 7544 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 00:14:42.979769 7544 mini_ranger_kms.cc:401] 127.7.94.20:33293/kms/v1/keys
Aug 11 00:14:43 dist-test-slave-8hhm krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407282, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.94.20@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
Aug 11 00:14:45 dist-test-slave-8hhm krb5kdc[26495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:14:45.691474 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:38719
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:38397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:38719
--ranger_config_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.94.20:33293/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:14:46.113898 27348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:14:46.114272 27348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:14:46.114460 27348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:14:46.122572 27348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:14:46.122746 27348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:14:46.122841 27348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:14:46.122952 27348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:14:46.123059 27348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:14:46.133477 27348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:38397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.94.20:33293/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:38719
--ranger_config_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:38719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:14:46.134986 27348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:14:46.136258 27348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:14:46.152674 27363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:14:46.154896 27368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:14:46.155480 27362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:14:46.156916 27348 server_base.cc:1034] running on GCE node
I20260811 00:14:46.158205 27348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:14:46.160286 27348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:14:46.161652 27348 hybrid_clock.cc:648] HybridClock initialized: now 1786407286161521 us; error 30 us; skew 500 ppm
I20260811 00:14:46.162132 27348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:14:46 dist-test-slave-8hhm krb5kdc[26495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407286, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:14:46.166121 27348 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:14:46.167836 27348 webserver.cc:460] Webserver started at http://127.7.94.62:45607/ using document root <none> and password file <none>
I20260811 00:14:46.168514 27348 fs_manager.cc:359] Metadata directory not provided
I20260811 00:14:46.168632 27348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:14:46.168902 27348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:14:46 dist-test-slave-8hhm krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407286, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
I20260811 00:14:46.234740 27348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "117c65e5dd9844ababec9cbd931ab4b5"
format_stamp: "Formatted at 2026-08-11 00:14:46 on dist-test-slave-8hhm"
server_key: "0c2705f3fb31035d167d2fa3b4e309b1"
server_key_iv: "736d1b7428a16ffcae5d6ec3496c4c7b"
server_key_version: "kuduclusterkey@0"
I20260811 00:14:46.235724 27348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "117c65e5dd9844ababec9cbd931ab4b5"
format_stamp: "Formatted at 2026-08-11 00:14:46 on dist-test-slave-8hhm"
server_key: "0c2705f3fb31035d167d2fa3b4e309b1"
server_key_iv: "736d1b7428a16ffcae5d6ec3496c4c7b"
server_key_version: "kuduclusterkey@0"
I20260811 00:14:46.242337 27348 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260811 00:14:46.262090 27384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:14:46.263787 27348 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 00:14:46.264019 27348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "117c65e5dd9844ababec9cbd931ab4b5"
format_stamp: "Formatted at 2026-08-11 00:14:46 on dist-test-slave-8hhm"
server_key: "0c2705f3fb31035d167d2fa3b4e309b1"
server_key_iv: "736d1b7428a16ffcae5d6ec3496c4c7b"
server_key_version: "kuduclusterkey@0"
I20260811 00:14:46.264336 27348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:14:46.292224 27348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:14:46.297374 27348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:14:46.297735 27348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:14:46.330644 27348 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:38719
I20260811 00:14:46.330750 27470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:38719 every 8 connection(s)
I20260811 00:14:46.332168 27348 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:14:46.336544 27475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:14:46.340677 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 27348
I20260811 00:14:46.340883 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:14:46.341614 7544 external_mini_cluster.cc:1428] Setting key 260d2fd9d11b29773c5705899ec9239b
I20260811 00:14:46.349298 27475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5: Bootstrap starting.
I20260811 00:14:46.352887 27475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5: Neither blocks nor log segments found. Creating new log.
I20260811 00:14:46.354151 27475 log.cc:824] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:14:46 dist-test-slave-8hhm krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:14:46.358047 27475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5: No bootstrap required, opened a new log
I20260811 00:14:46.366384 27485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:46.343889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54860 (local address 127.7.94.62:38719)
0811 00:14:46.344854 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0811 00:14:46.344864 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:14:46.344906 (+ 42us) server_negotiation.cc:407] Connection header received
0811 00:14:46.347171 (+ 2265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:14:46.347193 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:14:46.347543 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:14:46.347840 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:14:46.349533 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:46.350810 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:46.352669 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:46.353436 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:46.353562 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:46.358098 (+ 4536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:14:46.358106 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:14:46.358119 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:14:46.358194 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:14:46.362263 (+ 4069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:14:46.362945 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:14:46.362953 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:14:46.362971 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:14:46.363149 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:14:46.363553 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:14:46.363558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:14:46.363563 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:14:46.364103 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:14:46.364453 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0811 00:14:46.364951 (+ 498us) server_negotiation.cc:299] Negotiation successful
0811 00:14:46.365365 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":431,"threads_started":1}
I20260811 00:14:46.367729 27475 raft_consensus.cc:348] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "117c65e5dd9844ababec9cbd931ab4b5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 38719 } }
I20260811 00:14:46.368295 27475 raft_consensus.cc:374] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:14:46.368480 27475 raft_consensus.cc:729] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117c65e5dd9844ababec9cbd931ab4b5, State: Initialized, Role: FOLLOWER
I20260811 00:14:46.369203 27475 consensus_queue.cc:260] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "117c65e5dd9844ababec9cbd931ab4b5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 38719 } }
I20260811 00:14:46.369603 27475 raft_consensus.cc:388] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:14:46.369750 27475 raft_consensus.cc:482] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:14:46.369884 27475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:14:46.372555 27475 raft_consensus.cc:504] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "117c65e5dd9844ababec9cbd931ab4b5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 38719 } }
I20260811 00:14:46.373188 27475 leader_election.cc:302] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 117c65e5dd9844ababec9cbd931ab4b5; no voters:
I20260811 00:14:46.373927 27475 leader_election.cc:288] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:14:46.374054 27487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:14:46.374975 27487 raft_consensus.cc:686] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5 [term 1 LEADER]: Becoming Leader. State: Replica: 117c65e5dd9844ababec9cbd931ab4b5, State: Running, Role: LEADER
I20260811 00:14:46.375615 27487 consensus_queue.cc:237] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "117c65e5dd9844ababec9cbd931ab4b5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 38719 } }
I20260811 00:14:46.376112 27475 sys_catalog.cc:564] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:14:46.378314 27488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "117c65e5dd9844ababec9cbd931ab4b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "117c65e5dd9844ababec9cbd931ab4b5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 38719 } } }
I20260811 00:14:46.378693 27488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5 [sys.catalog]: This master's current role is: LEADER
I20260811 00:14:46.379177 27491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 117c65e5dd9844ababec9cbd931ab4b5. Latest consensus state: current_term: 1 leader_uuid: "117c65e5dd9844ababec9cbd931ab4b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "117c65e5dd9844ababec9cbd931ab4b5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 38719 } } }
I20260811 00:14:46.379595 27491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5 [sys.catalog]: This master's current role is: LEADER
I20260811 00:14:46.381759 27500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:14:46.390046 27500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:14:46.390883 27475 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 00:14:46.415516 27500 catalog_manager.cc:1347] Generated new cluster ID: c2ad0c7ed99e483ea3be5421822b77a3
I20260811 00:14:46.415681 27500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:14:46.437901 27500 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:14:46.439896 27500 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:14:46.455639 27500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5: Generated new TSK 0
I20260811 00:14:46.456769 27500 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
I20260811 00:14:47.341168 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.94.20:33293/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:38719
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:38397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:14:47 dist-test-slave-8hhm krb5kdc[26495](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786407287, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 00:14:47.900704 27560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:14:47.901037 27560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:14:47.901198 27560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:14:47.912209 27560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:14:47.912375 27560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:14:47.912587 27560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:14:47.925740 27560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:38397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.94.20:33293/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:38719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:14:47.927482 27560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:14:47.929194 27560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:14:47.950328 27578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:14:47.950870 27583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:14:47.953591 27577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:14:47.955291 27560 server_base.cc:1034] running on GCE node
I20260811 00:14:47.956012 27560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:14:47.957351 27560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:14:47.958590 27560 hybrid_clock.cc:648] HybridClock initialized: now 1786407287958505 us; error 91 us; skew 500 ppm
I20260811 00:14:47.958936 27560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:14:47 dist-test-slave-8hhm krb5kdc[26495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407287, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:14:47.963418 27560 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:14:47.965842 27560 webserver.cc:460] Webserver started at http://127.7.94.1:34299/ using document root <none> and password file <none>
I20260811 00:14:47.966955 27560 fs_manager.cc:359] Metadata directory not provided
I20260811 00:14:47.967129 27560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:14:47.967545 27560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:14:47 dist-test-slave-8hhm krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407287, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
I20260811 00:14:47.990010 27560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "4c4f15675c4c48f78e24520443a0a3e1"
format_stamp: "Formatted at 2026-08-11 00:14:47 on dist-test-slave-8hhm"
server_key: "28f2d16e44bd15587568e66273d6a47f"
server_key_iv: "20f4c5e7e645e7763aa4e39b9b509947"
server_key_version: "kuduclusterkey@0"
I20260811 00:14:47.991467 27560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "4c4f15675c4c48f78e24520443a0a3e1"
format_stamp: "Formatted at 2026-08-11 00:14:47 on dist-test-slave-8hhm"
server_key: "28f2d16e44bd15587568e66273d6a47f"
server_key_iv: "20f4c5e7e645e7763aa4e39b9b509947"
server_key_version: "kuduclusterkey@0"
I20260811 00:14:48.019799 27560 fs_manager.cc:691] Time spent creating directory manager: real 0.028s user 0.021s sys 0.008s
I20260811 00:14:48.051069 27594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:14:48.052891 27560 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.007s
I20260811 00:14:48.053093 27560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "4c4f15675c4c48f78e24520443a0a3e1"
format_stamp: "Formatted at 2026-08-11 00:14:47 on dist-test-slave-8hhm"
server_key: "28f2d16e44bd15587568e66273d6a47f"
server_key_iv: "20f4c5e7e645e7763aa4e39b9b509947"
server_key_version: "kuduclusterkey@0"
I20260811 00:14:48.053396 27560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:14:48.100662 27560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:14:48.105707 27560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:14:48.106048 27560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:14:48.110659 27560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:14:48.113312 27560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:14:48.113481 27560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260811 00:14:48.113646 27560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:14:48.113786 27560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:14:48 dist-test-slave-8hhm krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407287, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
I20260811 00:14:48.283495 27560 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:43193
I20260811 00:14:48.283495 27814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:43193 every 8 connection(s)
I20260811 00:14:48.285369 27560 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:14:48.287338 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 27560
I20260811 00:14:48.287626 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:14:48.288748 7544 external_mini_cluster.cc:1428] Setting key 02d8fb446e973f725f42cc4859fc8e55
Aug 11 00:14:48 dist-test-slave-8hhm krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407287, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:14:48.357275 27826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:48.306284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:38719 (local address 127.7.94.1:43721)
0811 00:14:48.320325 (+ 14041us) negotiation.cc:107] Waiting for socket to connect
0811 00:14:48.320412 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0811 00:14:48.326141 (+ 5729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:14:48.331731 (+ 5590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:14:48.331745 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:14:48.332456 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:14:48.333541 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:14:48.333566 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:14:48.335583 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:14:48.335589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:14:48.337997 (+ 2408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:14:48.338029 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:14:48.339492 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:14:48.339499 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:14:48.339898 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:48.341091 (+ 1193us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:14:48.341127 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:14:48.345883 (+ 4756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:14:48.351517 (+ 5634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:14:48.351530 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:14:48.351551 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:14:48.352425 (+ 874us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:14:48.353073 (+ 648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:14:48.353081 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:14:48.353087 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:14:48.353459 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:14:48.354247 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:14:48.354258 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:14:48.354735 (+ 477us) client_negotiation.cc:769] Sending connection context
0811 00:14:48.355097 (+ 362us) client_negotiation.cc:240] Negotiation successful
0811 00:14:48.355686 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13549,"thread_start_us":14772,"threads_started":1}
I20260811 00:14:48.359047 27824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:48.305545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:43721 (local address 127.7.94.62:38719)
0811 00:14:48.330655 (+ 25110us) server_negotiation.cc:206] Beginning negotiation
0811 00:14:48.330662 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:14:48.330690 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:14:48.330755 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:14:48.330760 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:14:48.330945 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:14:48.333729 (+ 2784us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:14:48.333917 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:48.335360 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:48.338187 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:48.338956 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:48.339080 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:48.346174 (+ 7094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:14:48.346182 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:14:48.346190 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:14:48.346235 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:14:48.351337 (+ 5102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:14:48.352574 (+ 1237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:14:48.352583 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:14:48.352589 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:14:48.352878 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:14:48.353603 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:14:48.353610 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:14:48.353616 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:14:48.354017 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:14:48.354173 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 00:14:48.358100 (+ 3927us) server_negotiation.cc:299] Negotiation successful
0811 00:14:48.358373 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24894,"thread_start_us":25592,"threads_started":1}
I20260811 00:14:48.362040 27818 heartbeater.cc:347] Connected to a master server at 127.7.94.62:38719
I20260811 00:14:48.362494 27818 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.94.2@KRBTEST.COM; defaulting to no policy
I20260811 00:14:48.363618 27818 heartbeater.cc:511] Master 127.7.94.62:38719 requested a full tablet report, sending...
I20260811 00:14:48.366158 27412 ts_manager.cc:194] Registered new tserver with Master: 4c4f15675c4c48f78e24520443a0a3e1 (127.7.94.1:43193)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.2@KRBTEST.COM" created.
I20260811 00:14:48.368232 27412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:43721
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
W20260811 00:14:48.401432 7544 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.7.94.2: real 0.109s user 0.000s sys 0.026s
I20260811 00:14:48.402233 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.94.20:33293/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:38719
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:38397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:14:48 dist-test-slave-8hhm krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407287, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
W20260811 00:14:48.956110 27834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:14:48.956403 27834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:14:48.956606 27834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:14:48.965899 27834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:14:48.966050 27834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:14:48.966284 27834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:14:48.978727 27834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:38397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.94.20:33293/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:38719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:14:48.980252 27834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:14:48.981755 27834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:14:49.001672 27873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:14:49.003003 27874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:14:49.003093 27878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:14:49.003782 27834 server_base.cc:1034] running on GCE node
I20260811 00:14:49.004521 27834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:14:49.005791 27834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:14:49.007077 27834 hybrid_clock.cc:648] HybridClock initialized: now 1786407289006946 us; error 77 us; skew 500 ppm
I20260811 00:14:49.007551 27834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:14:49 dist-test-slave-8hhm krb5kdc[26495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407289, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:14:49.013574 27834 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:14:49.015610 27834 webserver.cc:460] Webserver started at http://127.7.94.2:32869/ using document root <none> and password file <none>
I20260811 00:14:49.016484 27834 fs_manager.cc:359] Metadata directory not provided
I20260811 00:14:49.016618 27834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:14:49.016954 27834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:14:49 dist-test-slave-8hhm krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407289, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
I20260811 00:14:49.035648 27834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "20e10de97d774d8fb89bcd367fa81150"
format_stamp: "Formatted at 2026-08-11 00:14:49 on dist-test-slave-8hhm"
server_key: "ff79871392bf07530daf7f25ca3e7c81"
server_key_iv: "d51a30f0e79adc1f1109969a86a6dc08"
server_key_version: "kuduclusterkey@0"
I20260811 00:14:49.037075 27834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "20e10de97d774d8fb89bcd367fa81150"
format_stamp: "Formatted at 2026-08-11 00:14:49 on dist-test-slave-8hhm"
server_key: "ff79871392bf07530daf7f25ca3e7c81"
server_key_iv: "d51a30f0e79adc1f1109969a86a6dc08"
server_key_version: "kuduclusterkey@0"
I20260811 00:14:49.047026 27834 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.008s sys 0.000s
I20260811 00:14:49.065321 27885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:14:49.066974 27834 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260811 00:14:49.067174 27834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "20e10de97d774d8fb89bcd367fa81150"
format_stamp: "Formatted at 2026-08-11 00:14:49 on dist-test-slave-8hhm"
server_key: "ff79871392bf07530daf7f25ca3e7c81"
server_key_iv: "d51a30f0e79adc1f1109969a86a6dc08"
server_key_version: "kuduclusterkey@0"
I20260811 00:14:49.067431 27834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:14:49.097311 27834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:14:49.101831 27834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:14:49.102250 27834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:14:49.103880 27834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:14:49.105901 27834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:14:49.106004 27834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:14:49.106168 27834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:14:49.106245 27834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:14:49.179565 27834 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:38175
I20260811 00:14:49.179744 28099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:38175 every 8 connection(s)
I20260811 00:14:49.181007 27834 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:14:49.189662 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 27834
I20260811 00:14:49.189864 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:14:49.190336 7544 external_mini_cluster.cc:1428] Setting key d553ad39b8952d792785550fe01456ab
Aug 11 00:14:49 dist-test-slave-8hhm krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407289, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:14:49.209916 28106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:49.184993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:48037 (local address 127.7.94.62:38719)
0811 00:14:49.186015 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0811 00:14:49.186023 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:14:49.187145 (+ 1122us) server_negotiation.cc:407] Connection header received
0811 00:14:49.192250 (+ 5105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:14:49.192255 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:14:49.192335 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:14:49.192470 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:14:49.194546 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:49.195312 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:49.197093 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:49.197688 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:49.197852 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:49.202401 (+ 4549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:14:49.202408 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:14:49.202412 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:14:49.202480 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:14:49.205811 (+ 3331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:14:49.206804 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:14:49.206810 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:14:49.206815 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:14:49.206978 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:14:49.207725 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:14:49.207737 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:14:49.207740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:14:49.208080 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:14:49.208286 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 00:14:49.209149 (+ 863us) server_negotiation.cc:299] Negotiation successful
0811 00:14:49.209431 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":791,"thread_start_us":619,"threads_started":1}
I20260811 00:14:49.211318 28108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:49.185831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:38719 (local address 127.7.94.2:48037)
0811 00:14:49.186881 (+ 1050us) negotiation.cc:107] Waiting for socket to connect
0811 00:14:49.186940 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 00:14:49.190903 (+ 3963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:14:49.192740 (+ 1837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:14:49.192749 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:14:49.193520 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:14:49.194357 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:14:49.194379 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:14:49.195501 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:14:49.195505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:14:49.196796 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:14:49.196816 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:14:49.197982 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:14:49.197991 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:14:49.198268 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:49.199137 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:14:49.199171 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:14:49.202093 (+ 2922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:14:49.205986 (+ 3893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:14:49.205996 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:14:49.206013 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:14:49.206637 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:14:49.207122 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:14:49.207128 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:14:49.207132 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:14:49.207571 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:14:49.208246 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:14:49.208255 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:14:49.208693 (+ 438us) client_negotiation.cc:769] Sending connection context
0811 00:14:49.209058 (+ 365us) client_negotiation.cc:240] Negotiation successful
0811 00:14:49.209713 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":414,"threads_started":1}
I20260811 00:14:49.213069 28101 heartbeater.cc:347] Connected to a master server at 127.7.94.62:38719
I20260811 00:14:49.213474 28101 heartbeater.cc:464] Registering TS with master...
I20260811 00:14:49.214277 28101 heartbeater.cc:511] Master 127.7.94.62:38719 requested a full tablet report, sending...
I20260811 00:14:49.215997 27412 ts_manager.cc:194] Registered new tserver with Master: 20e10de97d774d8fb89bcd367fa81150 (127.7.94.2:38175)
I20260811 00:14:49.216936 27412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:48037
WARNING: no policy specified for kudu/127.7.94.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
I20260811 00:14:49.266768 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.94.20:33293/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:38719
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:38397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:14:49.372888 27818 heartbeater.cc:503] Master 127.7.94.62:38719 was elected leader, sending a full tablet report...
W20260811 00:14:49.730055 28115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:14:49.730407 28115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:14:49.730598 28115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:14:49.739194 28115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:14:49.739354 28115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:14:49.739531 28115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:14:49.750094 28115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:38397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.94.20:33293/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:38719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:14:49.751595 28115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:14:49.753053 28115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:14:49.766842 28120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:14:49.766784 28126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:14:49.767163 28123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:14:49.768293 28115 server_base.cc:1034] running on GCE node
I20260811 00:14:49.768996 28115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:14:49.770300 28115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:14:49.771574 28115 hybrid_clock.cc:648] HybridClock initialized: now 1786407289771483 us; error 53 us; skew 500 ppm
I20260811 00:14:49.772043 28115 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:14:49 dist-test-slave-8hhm krb5kdc[26495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407289, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:14:49.775691 28115 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:14:49.777266 28115 webserver.cc:460] Webserver started at http://127.7.94.3:34949/ using document root <none> and password file <none>
I20260811 00:14:49.777990 28115 fs_manager.cc:359] Metadata directory not provided
I20260811 00:14:49.778148 28115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:14:49.778472 28115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:14:49 dist-test-slave-8hhm krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407289, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
I20260811 00:14:49.793676 28115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "3c39d9511f314b708305be8a3507429f"
format_stamp: "Formatted at 2026-08-11 00:14:49 on dist-test-slave-8hhm"
server_key: "9d34ded92bd1e029fe2073c2ac20991f"
server_key_iv: "37aa117ce95699836b65272f8ae8f93a"
server_key_version: "kuduclusterkey@0"
I20260811 00:14:49.794538 28115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "3c39d9511f314b708305be8a3507429f"
format_stamp: "Formatted at 2026-08-11 00:14:49 on dist-test-slave-8hhm"
server_key: "9d34ded92bd1e029fe2073c2ac20991f"
server_key_iv: "37aa117ce95699836b65272f8ae8f93a"
server_key_version: "kuduclusterkey@0"
I20260811 00:14:49.800669 28115 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260811 00:14:49.816903 28139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:14:49.818818 28115 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 00:14:49.819087 28115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "3c39d9511f314b708305be8a3507429f"
format_stamp: "Formatted at 2026-08-11 00:14:49 on dist-test-slave-8hhm"
server_key: "9d34ded92bd1e029fe2073c2ac20991f"
server_key_iv: "37aa117ce95699836b65272f8ae8f93a"
server_key_version: "kuduclusterkey@0"
I20260811 00:14:49.819419 28115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:14:49.848055 28115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:14:49.852475 28115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:14:49.852784 28115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:14:49.853920 28115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:14:49.855334 28115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:14:49.855443 28115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:14:49.855595 28115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:14:49.855674 28115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:14:49.907160 28115 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:33999
I20260811 00:14:49.907351 28379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:33999 every 8 connection(s)
I20260811 00:14:49.908672 28115 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:14:49.909467 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 28115
I20260811 00:14:49.909677 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:14:49.910367 7544 external_mini_cluster.cc:1428] Setting key b71ef4f301fbca03d40a59e8860ab335
Aug 11 00:14:49 dist-test-slave-8hhm krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407289, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:14:49.930998 28384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:49.912197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:38289 (local address 127.7.94.62:38719)
0811 00:14:49.913105 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0811 00:14:49.913111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:14:49.914796 (+ 1685us) server_negotiation.cc:407] Connection header received
0811 00:14:49.916635 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:14:49.916639 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:14:49.916741 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:14:49.916829 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:14:49.918478 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:49.919284 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:49.920783 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:49.921573 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:49.921683 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:49.925269 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:14:49.925277 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:14:49.925281 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:14:49.925320 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:14:49.928378 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:14:49.929033 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:14:49.929036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:14:49.929038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:14:49.929167 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:14:49.929632 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:14:49.929636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:14:49.929637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:14:49.929881 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:14:49.929997 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 00:14:49.930607 (+ 610us) server_negotiation.cc:299] Negotiation successful
0811 00:14:49.930759 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":759,"thread_start_us":554,"threads_started":1}
I20260811 00:14:49.931890 28388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:49.913169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:38719 (local address 127.7.94.3:38289)
0811 00:14:49.914627 (+ 1458us) negotiation.cc:107] Waiting for socket to connect
0811 00:14:49.914688 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 00:14:49.916412 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:14:49.917028 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:14:49.917035 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:14:49.917557 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:14:49.918330 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:14:49.918351 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:14:49.919438 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:14:49.919441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:14:49.920640 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:14:49.920660 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:14:49.921773 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:14:49.921789 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:14:49.921993 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:49.922712 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:14:49.922736 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:14:49.924980 (+ 2244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:14:49.928505 (+ 3525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:14:49.928512 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:14:49.928521 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:14:49.928936 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:14:49.929344 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:14:49.929347 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:14:49.929349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:14:49.929528 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:14:49.929993 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:14:49.929998 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:14:49.930270 (+ 272us) client_negotiation.cc:769] Sending connection context
0811 00:14:49.930529 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 00:14:49.930863 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1170,"thread_start_us":781,"threads_started":1}
I20260811 00:14:49.933343 28380 heartbeater.cc:347] Connected to a master server at 127.7.94.62:38719
I20260811 00:14:49.933705 28380 heartbeater.cc:464] Registering TS with master...
I20260811 00:14:49.934542 28380 heartbeater.cc:511] Master 127.7.94.62:38719 requested a full tablet report, sending...
I20260811 00:14:49.935906 27412 ts_manager.cc:194] Registered new tserver with Master: 3c39d9511f314b708305be8a3507429f (127.7.94.3:33999)
I20260811 00:14:49.936668 27412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:38289
I20260811 00:14:49.938665 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:14:49.957348 28384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:49.943475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54866 (local address 127.7.94.62:38719)
0811 00:14:49.943595 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 00:14:49.943606 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:14:49.944314 (+ 708us) server_negotiation.cc:407] Connection header received
0811 00:14:49.944519 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:14:49.944530 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:14:49.944634 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:14:49.944737 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:14:49.946260 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:49.947052 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:49.948662 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:49.949306 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:49.949439 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:49.951691 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:14:49.951706 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:14:49.951709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:14:49.951737 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:14:49.954275 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:14:49.954893 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:14:49.954900 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:14:49.954903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:14:49.955023 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:14:49.955482 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:14:49.955487 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:14:49.955491 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:14:49.955745 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:14:49.956041 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 00:14:49.956594 (+ 553us) server_negotiation.cc:299] Negotiation successful
0811 00:14:49.956808 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 00:14:49.961783 27412 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:14:49.969424 27412 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:14:50.006847 28404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:49.982997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:33999 (local address 127.0.0.1:50348)
0811 00:14:49.985812 (+ 2815us) negotiation.cc:107] Waiting for socket to connect
0811 00:14:49.985832 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:14:49.985962 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:14:49.996910 (+ 10948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:14:49.996921 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:14:49.996961 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:14:49.997540 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:14:49.997557 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.000275 (+ 2718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.000283 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:14:50.002927 (+ 2644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:14:50.002939 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.005634 (+ 2695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.005642 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:14:50.005904 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:50.005955 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 00:14:50.006117 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 00:14:50.006324 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2668,"thread_start_us":2488,"threads_started":1}
I20260811 00:14:50.008059 28405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:49.983889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50348 (local address 127.7.94.3:33999)
0811 00:14:49.994657 (+ 10768us) server_negotiation.cc:206] Beginning negotiation
0811 00:14:49.994665 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:14:49.994700 (+ 35us) server_negotiation.cc:407] Connection header received
0811 00:14:49.994824 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:14:49.994836 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:14:49.995166 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:14:49.997814 (+ 2648us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:14:49.998185 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.000045 (+ 1860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.003100 (+ 3055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.004922 (+ 1822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.005043 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:50.005386 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0811 00:14:50.007454 (+ 2068us) server_negotiation.cc:299] Negotiation successful
0811 00:14:50.007647 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10594,"thread_start_us":11507,"threads_started":1}
I20260811 00:14:50.008231 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:49.986693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:38175 (local address 127.0.0.1:53436)
0811 00:14:49.999720 (+ 13027us) negotiation.cc:107] Waiting for socket to connect
0811 00:14:49.999745 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:14:49.999910 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:14:50.001066 (+ 1156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:14:50.001083 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:14:50.001125 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:14:50.001814 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:14:50.001829 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.003662 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.003668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:14:50.005922 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:14:50.005934 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.007467 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.007472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:14:50.007583 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:50.007607 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 00:14:50.007702 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 00:14:50.007826 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12880,"mutex_wait_us":34,"thread_start_us":12752,"threads_started":1}
I20260811 00:14:50.008556 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:49.987059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53436 (local address 127.7.94.2:38175)
0811 00:14:50.000303 (+ 13244us) server_negotiation.cc:206] Beginning negotiation
0811 00:14:50.000313 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:14:50.000342 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:14:50.000518 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:14:50.000526 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:14:50.000758 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:14:50.001058 (+ 300us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:14:50.002076 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.003518 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.006075 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.007042 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.007176 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:50.007337 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 00:14:50.007781 (+ 444us) server_negotiation.cc:299] Negotiation successful
0811 00:14:50.007945 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13067,"thread_start_us":12868,"threads_started":1}
I20260811 00:14:50.008898 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:49.982936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:43193 (local address 127.0.0.1:48460)
0811 00:14:49.983797 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0811 00:14:49.983848 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 00:14:49.984060 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:14:50.000147 (+ 16087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:14:50.000152 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:14:50.000179 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:14:50.001274 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:14:50.001293 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.003345 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.003350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:14:50.006149 (+ 2799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:14:50.006162 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.008135 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.008142 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:14:50.008345 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:50.008386 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 00:14:50.008453 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 00:14:50.008577 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":1367,"threads_started":1}
I20260811 00:14:50.010123 28408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:49.984012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48460 (local address 127.7.94.1:43193)
0811 00:14:49.996167 (+ 12155us) server_negotiation.cc:206] Beginning negotiation
0811 00:14:49.996180 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:14:49.996214 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:14:49.996330 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:14:49.996341 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:14:49.996618 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:14:50.001383 (+ 4765us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:14:50.001659 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.003196 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.006317 (+ 3121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.007872 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.009328 (+ 1456us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:50.009501 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 00:14:50.009559 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 00:14:50.009659 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11919,"thread_start_us":15033,"threads_started":1}
I20260811 00:14:50.012537 27976 tablet_service.cc:1467] Processing CreateTablet for tablet 3e825632867342d2aba9c527c4ff7be7 (DEFAULT_TABLE table=test-table [id=acce764e58a44e279e3f8c92b068c44c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:14:50.012948 28242 tablet_service.cc:1467] Processing CreateTablet for tablet 3e825632867342d2aba9c527c4ff7be7 (DEFAULT_TABLE table=test-table [id=acce764e58a44e279e3f8c92b068c44c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:14:50.013876 27976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e825632867342d2aba9c527c4ff7be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:14:50.014012 27684 tablet_service.cc:1467] Processing CreateTablet for tablet 3e825632867342d2aba9c527c4ff7be7 (DEFAULT_TABLE table=test-table [id=acce764e58a44e279e3f8c92b068c44c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:14:50.014696 28242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e825632867342d2aba9c527c4ff7be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:14:50.015556 27684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e825632867342d2aba9c527c4ff7be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:14:50.026134 28413 tablet_bootstrap.cc:492] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150: Bootstrap starting.
I20260811 00:14:50.029554 28413 tablet_bootstrap.cc:654] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150: Neither blocks nor log segments found. Creating new log.
I20260811 00:14:50.031489 28413 log.cc:824] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150: Log is configured to *not* fsync() on all Append() calls
I20260811 00:14:50.034763 28413 tablet_bootstrap.cc:492] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150: No bootstrap required, opened a new log
I20260811 00:14:50.035055 28413 ts_tablet_manager.cc:1397] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 00:14:50.035108 28419 tablet_bootstrap.cc:492] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f: Bootstrap starting.
I20260811 00:14:50.038389 28417 tablet_bootstrap.cc:492] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1: Bootstrap starting.
I20260811 00:14:50.039602 28419 tablet_bootstrap.cc:654] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f: Neither blocks nor log segments found. Creating new log.
I20260811 00:14:50.041183 28419 log.cc:824] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:14:50.042809 28417 tablet_bootstrap.cc:654] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1: Neither blocks nor log segments found. Creating new log.
I20260811 00:14:50.044605 28417 log.cc:824] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1: Log is configured to *not* fsync() on all Append() calls
I20260811 00:14:50.045544 28419 tablet_bootstrap.cc:492] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f: No bootstrap required, opened a new log
I20260811 00:14:50.045807 28419 ts_tablet_manager.cc:1397] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20260811 00:14:50.047150 28417 tablet_bootstrap.cc:492] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1: No bootstrap required, opened a new log
I20260811 00:14:50.047399 28417 ts_tablet_manager.cc:1397] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.001s
I20260811 00:14:50.047266 28413 raft_consensus.cc:348] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c39d9511f314b708305be8a3507429f" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33999 } } peers { permanent_uuid: "4c4f15675c4c48f78e24520443a0a3e1" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 43193 } } peers { permanent_uuid: "20e10de97d774d8fb89bcd367fa81150" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38175 } }
I20260811 00:14:50.047703 28413 raft_consensus.cc:374] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:14:50.047837 28413 raft_consensus.cc:729] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20e10de97d774d8fb89bcd367fa81150, State: Initialized, Role: FOLLOWER
I20260811 00:14:50.048494 28413 consensus_queue.cc:260] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c39d9511f314b708305be8a3507429f" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33999 } } peers { permanent_uuid: "4c4f15675c4c48f78e24520443a0a3e1" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 43193 } } peers { permanent_uuid: "20e10de97d774d8fb89bcd367fa81150" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38175 } }
I20260811 00:14:50.050081 28101 heartbeater.cc:503] Master 127.7.94.62:38719 was elected leader, sending a full tablet report...
I20260811 00:14:50.052047 28413 ts_tablet_manager.cc:1428] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150: Time spent starting tablet: real 0.017s user 0.007s sys 0.007s
I20260811 00:14:50.057484 28417 raft_consensus.cc:348] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c39d9511f314b708305be8a3507429f" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33999 } } peers { permanent_uuid: "4c4f15675c4c48f78e24520443a0a3e1" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 43193 } } peers { permanent_uuid: "20e10de97d774d8fb89bcd367fa81150" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38175 } }
I20260811 00:14:50.057853 28417 raft_consensus.cc:374] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:14:50.057963 28417 raft_consensus.cc:729] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c4f15675c4c48f78e24520443a0a3e1, State: Initialized, Role: FOLLOWER
I20260811 00:14:50.058528 28417 consensus_queue.cc:260] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c39d9511f314b708305be8a3507429f" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33999 } } peers { permanent_uuid: "4c4f15675c4c48f78e24520443a0a3e1" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 43193 } } peers { permanent_uuid: "20e10de97d774d8fb89bcd367fa81150" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38175 } }
I20260811 00:14:50.058483 28419 raft_consensus.cc:348] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c39d9511f314b708305be8a3507429f" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33999 } } peers { permanent_uuid: "4c4f15675c4c48f78e24520443a0a3e1" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 43193 } } peers { permanent_uuid: "20e10de97d774d8fb89bcd367fa81150" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38175 } }
I20260811 00:14:50.058969 28419 raft_consensus.cc:374] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:14:50.059162 28419 raft_consensus.cc:729] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c39d9511f314b708305be8a3507429f, State: Initialized, Role: FOLLOWER
I20260811 00:14:50.059809 28419 consensus_queue.cc:260] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c39d9511f314b708305be8a3507429f" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33999 } } peers { permanent_uuid: "4c4f15675c4c48f78e24520443a0a3e1" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 43193 } } peers { permanent_uuid: "20e10de97d774d8fb89bcd367fa81150" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38175 } }
I20260811 00:14:50.060642 28417 ts_tablet_manager.cc:1428] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1: Time spent starting tablet: real 0.013s user 0.011s sys 0.004s
I20260811 00:14:50.062378 28380 heartbeater.cc:503] Master 127.7.94.62:38719 was elected leader, sending a full tablet report...
I20260811 00:14:50.063083 28419 ts_tablet_manager.cc:1428] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f: Time spent starting tablet: real 0.017s user 0.014s sys 0.001s
W20260811 00:14:50.089833 27819 tablet.cc:2369] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:14:50.153596 28428 raft_consensus.cc:482] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:14:50.153970 28428 raft_consensus.cc:504] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c39d9511f314b708305be8a3507429f" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33999 } } peers { permanent_uuid: "4c4f15675c4c48f78e24520443a0a3e1" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 43193 } } peers { permanent_uuid: "20e10de97d774d8fb89bcd367fa81150" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38175 } }
I20260811 00:14:50.156239 28428 leader_election.cc:288] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c4f15675c4c48f78e24520443a0a3e1 (127.7.94.1:43193), 20e10de97d774d8fb89bcd367fa81150 (127.7.94.2:38175)
W20260811 00:14:50.161270 28381 tablet.cc:2369] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:14:50.164302 28408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:50.157024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:35069 (local address 127.7.94.1:43193)
0811 00:14:50.157417 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0811 00:14:50.157425 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:14:50.157449 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:14:50.157537 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:14:50.157544 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:14:50.157630 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:14:50.157740 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:14:50.158184 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.159704 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.162434 (+ 2730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.163436 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.163533 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:50.163613 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 00:14:50.164002 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 00:14:50.164091 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260811 00:14:50.164498 28388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:50.156505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:43193 (local address 127.7.94.3:35069)
0811 00:14:50.156813 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 00:14:50.156856 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 00:14:50.157013 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:14:50.157768 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:14:50.157772 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:14:50.157801 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:14:50.158020 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:14:50.158030 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.159849 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.159855 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:14:50.162278 (+ 2423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:14:50.162294 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.163582 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.163588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:14:50.163805 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:50.163849 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 00:14:50.163974 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 00:14:50.164160 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":51}
I20260811 00:14:50.165746 28429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:50.156848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:38175 (local address 127.7.94.3:38341)
0811 00:14:50.157827 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0811 00:14:50.157849 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:14:50.158002 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:14:50.158374 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:14:50.158379 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:14:50.158415 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:14:50.159125 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:14:50.159140 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.160577 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.160581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:14:50.162862 (+ 2281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:14:50.162877 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.164950 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.164965 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:14:50.165190 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:50.165249 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 00:14:50.165341 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 00:14:50.165428 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":844,"mutex_wait_us":1,"thread_start_us":629,"threads_started":1}
I20260811 00:14:50.165791 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:50.157390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:38341 (local address 127.7.94.2:38175)
0811 00:14:50.157617 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 00:14:50.157623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:14:50.157956 (+ 333us) server_negotiation.cc:407] Connection header received
0811 00:14:50.158137 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:14:50.158142 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:14:50.158227 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:14:50.158296 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:14:50.159350 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.160373 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.163040 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.164719 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.164860 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:50.164925 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 00:14:50.165406 (+ 481us) server_negotiation.cc:299] Negotiation successful
0811 00:14:50.165542 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 00:14:50.165872 27724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e825632867342d2aba9c527c4ff7be7" candidate_uuid: "3c39d9511f314b708305be8a3507429f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c4f15675c4c48f78e24520443a0a3e1" is_pre_election: true
I20260811 00:14:50.166549 27724 raft_consensus.cc:2454] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c39d9511f314b708305be8a3507429f in term 0.
I20260811 00:14:50.167328 28150 leader_election.cc:302] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c39d9511f314b708305be8a3507429f, 4c4f15675c4c48f78e24520443a0a3e1; no voters:
I20260811 00:14:50.167383 28018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e825632867342d2aba9c527c4ff7be7" candidate_uuid: "3c39d9511f314b708305be8a3507429f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20e10de97d774d8fb89bcd367fa81150" is_pre_election: true
I20260811 00:14:50.167732 28428 raft_consensus.cc:2781] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:14:50.167944 28428 raft_consensus.cc:482] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:14:50.168073 28428 raft_consensus.cc:3037] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:14:50.168232 28018 raft_consensus.cc:2454] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c39d9511f314b708305be8a3507429f in term 0.
I20260811 00:14:50.171736 28428 raft_consensus.cc:504] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c39d9511f314b708305be8a3507429f" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33999 } } peers { permanent_uuid: "4c4f15675c4c48f78e24520443a0a3e1" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 43193 } } peers { permanent_uuid: "20e10de97d774d8fb89bcd367fa81150" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38175 } }
I20260811 00:14:50.172753 28428 leader_election.cc:288] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [CANDIDATE]: Term 1 election: Requested vote from peers 4c4f15675c4c48f78e24520443a0a3e1 (127.7.94.1:43193), 20e10de97d774d8fb89bcd367fa81150 (127.7.94.2:38175)
I20260811 00:14:50.173277 27724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e825632867342d2aba9c527c4ff7be7" candidate_uuid: "3c39d9511f314b708305be8a3507429f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c4f15675c4c48f78e24520443a0a3e1"
I20260811 00:14:50.173352 28018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e825632867342d2aba9c527c4ff7be7" candidate_uuid: "3c39d9511f314b708305be8a3507429f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20e10de97d774d8fb89bcd367fa81150"
I20260811 00:14:50.173588 27724 raft_consensus.cc:3037] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:14:50.173573 28018 raft_consensus.cc:3037] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:14:50.177691 27724 raft_consensus.cc:2454] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c39d9511f314b708305be8a3507429f in term 1.
I20260811 00:14:50.177913 28018 raft_consensus.cc:2454] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c39d9511f314b708305be8a3507429f in term 1.
I20260811 00:14:50.178109 28150 leader_election.cc:302] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c39d9511f314b708305be8a3507429f, 4c4f15675c4c48f78e24520443a0a3e1; no voters:
I20260811 00:14:50.178421 28428 raft_consensus.cc:2781] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:14:50.179265 28428 raft_consensus.cc:686] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [term 1 LEADER]: Becoming Leader. State: Replica: 3c39d9511f314b708305be8a3507429f, State: Running, Role: LEADER
I20260811 00:14:50.179915 28428 consensus_queue.cc:237] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c39d9511f314b708305be8a3507429f" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33999 } } peers { permanent_uuid: "4c4f15675c4c48f78e24520443a0a3e1" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 43193 } } peers { permanent_uuid: "20e10de97d774d8fb89bcd367fa81150" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38175 } }
W20260811 00:14:50.184168 28103 tablet.cc:2369] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:14:50.185639 27412 catalog_manager.cc:5495] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c39d9511f314b708305be8a3507429f (127.7.94.3). New cstate: current_term: 1 leader_uuid: "3c39d9511f314b708305be8a3507429f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c39d9511f314b708305be8a3507429f" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c4f15675c4c48f78e24520443a0a3e1" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 43193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20e10de97d774d8fb89bcd367fa81150" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38175 } health_report { overall_health: UNKNOWN } } }
I20260811 00:14:50.230816 28405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:50.225529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50362 (local address 127.7.94.3:33999)
0811 00:14:50.225746 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 00:14:50.225753 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:14:50.225921 (+ 168us) server_negotiation.cc:407] Connection header received
0811 00:14:50.226077 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:14:50.226081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:14:50.226180 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:14:50.226303 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:14:50.226635 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.227367 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.228769 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.229162 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.229284 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:50.229749 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:14:50.230258 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:14:50.230377 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 00:14:50.230447 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 00:14:50.230511 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 00:14:50.240610 28018 raft_consensus.cc:1261] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150 [term 1 FOLLOWER]: Refusing update from remote peer 3c39d9511f314b708305be8a3507429f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:14:50.240741 27724 raft_consensus.cc:1261] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1 [term 1 FOLLOWER]: Refusing update from remote peer 3c39d9511f314b708305be8a3507429f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:14:50.241688 28431 consensus_queue.cc:1035] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [LEADER]: Connected to new peer: Peer: permanent_uuid: "20e10de97d774d8fb89bcd367fa81150" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 38175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:14:50.242137 28428 consensus_queue.cc:1035] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c4f15675c4c48f78e24520443a0a3e1" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 43193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:14:50.256182 28439 mvcc.cc:205] Tried to move back new op lower bound from 7317124260814483456 to 7317124260581744640. Current Snapshot: MvccSnapshot[applied={T|T < 7317124260814483456}]
I20260811 00:14:50.271663 27411 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:14:50.272250 27411 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:14:50.277370 27411 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 117c65e5dd9844ababec9cbd931ab4b5: Sending DeleteTablet for 3 replicas of tablet 3e825632867342d2aba9c527c4ff7be7
I20260811 00:14:50.278648 28242 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e825632867342d2aba9c527c4ff7be7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:14:50 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:50348
I20260811 00:14:50.278853 27684 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e825632867342d2aba9c527c4ff7be7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:14:50 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:48460
I20260811 00:14:50.279071 27976 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e825632867342d2aba9c527c4ff7be7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:14:50 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:53436
I20260811 00:14:50.281782 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 27560
I20260811 00:14:50.295666 28460 tablet_replica.cc:330] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150: stopping tablet replica
I20260811 00:14:50.296779 28460 raft_consensus.cc:2229] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:14:50.297966 28460 raft_consensus.cc:2258] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:14:50.298465 28459 tablet_replica.cc:330] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1: stopping tablet replica
I20260811 00:14:50.299137 28459 raft_consensus.cc:2229] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:14:50.299821 28459 raft_consensus.cc:2258] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:14:50.300439 28454 tablet_replica.cc:330] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f: stopping tablet replica
I20260811 00:14:50.300897 28460 ts_tablet_manager.cc:1905] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:14:50.301383 28454 raft_consensus.cc:2229] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:14:50.302116 28454 raft_consensus.cc:2258] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 11 00:14:50 dist-test-slave-8hhm krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:14:50.306854 28459 ts_tablet_manager.cc:1905] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:14:50.309276 28460 ts_tablet_manager.cc:1918] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:14:50.309541 28460 log.cc:1196] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/3e825632867342d2aba9c527c4ff7be7
I20260811 00:14:50.310231 28460 ts_tablet_manager.cc:1939] T 3e825632867342d2aba9c527c4ff7be7 P 20e10de97d774d8fb89bcd367fa81150: Deleting consensus metadata
I20260811 00:14:50.312348 27391 catalog_manager.cc:4841] TS 20e10de97d774d8fb89bcd367fa81150 (127.7.94.2:38175): tablet 3e825632867342d2aba9c527c4ff7be7 (table test-table [id=acce764e58a44e279e3f8c92b068c44c]) successfully deleted
I20260811 00:14:50.314904 28408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:50.283464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48466 (local address 127.7.94.1:43193)
0811 00:14:50.283687 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 00:14:50.283696 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:14:50.291537 (+ 7841us) server_negotiation.cc:407] Connection header received
0811 00:14:50.291782 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:14:50.291791 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:14:50.291926 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:14:50.292045 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:14:50.294229 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.295390 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.297607 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.298826 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.298927 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:50.305861 (+ 6934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:14:50.305875 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:14:50.305893 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:14:50.305942 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:14:50.310704 (+ 4762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:14:50.311724 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:14:50.311733 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:14:50.311737 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:14:50.311944 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:14:50.312431 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:14:50.312436 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:14:50.312440 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:14:50.312997 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:14:50.313381 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0811 00:14:50.313962 (+ 581us) server_negotiation.cc:299] Negotiation successful
0811 00:14:50.314237 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:14:50.315887 27804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:14:50.317867 28454 ts_tablet_manager.cc:1905] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:14:50.327325 28454 ts_tablet_manager.cc:1918] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:14:50.327630 28454 log.cc:1196] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/3e825632867342d2aba9c527c4ff7be7
I20260811 00:14:50.328418 28454 ts_tablet_manager.cc:1939] T 3e825632867342d2aba9c527c4ff7be7 P 3c39d9511f314b708305be8a3507429f: Deleting consensus metadata
I20260811 00:14:50.330749 27390 catalog_manager.cc:4841] TS 3c39d9511f314b708305be8a3507429f (127.7.94.3:33999): tablet 3e825632867342d2aba9c527c4ff7be7 (table test-table [id=acce764e58a44e279e3f8c92b068c44c]) successfully deleted
I20260811 00:14:50.431376 28459 ts_tablet_manager.cc:1918] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:14:50.431661 28459 log.cc:1196] T 3e825632867342d2aba9c527c4ff7be7 P 4c4f15675c4c48f78e24520443a0a3e1: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/3e825632867342d2aba9c527c4ff7be7
I20260811 00:14:50.432215 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 27560
W20260811 00:14:50.465683 27391 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.94.1:43193 (error 108)
I20260811 00:14:50.465929 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 27834
I20260811 00:14:50.467865 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:50.467328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:43193 (local address 127.0.0.1:48478)
0811 00:14:50.467577 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 00:14:50.467726 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:43193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 00:14:50.468178 27391 catalog_manager.cc:4584] TS 4c4f15675c4c48f78e24520443a0a3e1 (127.7.94.1:43193): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e825632867342d2aba9c527c4ff7be7: Network error: Client connection negotiation failed: client connection to 127.7.94.1:43193: connect: Connection refused (error 111)
Aug 11 00:14:50 dist-test-slave-8hhm krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.2@KRBTEST.COM
I20260811 00:14:50.484603 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:50.467464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53448 (local address 127.7.94.2:38175)
0811 00:14:50.467657 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 00:14:50.467665 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:14:50.467689 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:14:50.467764 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:14:50.467769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:14:50.467863 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:14:50.467967 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:14:50.469841 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.470683 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.472616 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.473038 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.473151 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:50.478479 (+ 5328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:14:50.478502 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:14:50.478518 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:14:50.478610 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:14:50.481627 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:14:50.482405 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:14:50.482412 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:14:50.482414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:14:50.482539 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:14:50.482936 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:14:50.482939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:14:50.482941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:14:50.483287 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:14:50.483569 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 00:14:50.483962 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 00:14:50.484198 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 00:14:50.485387 28092 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:14:50.522200 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:50.521668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:43193 (local address 127.0.0.1:48484)
0811 00:14:50.521915 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 00:14:50.522025 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:43193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 00:14:50.523407 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:50.523006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:43193 (local address 127.0.0.1:48494)
0811 00:14:50.523172 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 00:14:50.523260 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:43193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 00:14:50.578687 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 27834
I20260811 00:14:50.584239 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:50.583547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:43193 (local address 127.0.0.1:48510)
0811 00:14:50.583824 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 00:14:50.583973 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:43193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 00:14:50.590314 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:50.589713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:43193 (local address 127.0.0.1:48512)
0811 00:14:50.589933 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 00:14:50.590134 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:43193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 00:14:50.605991 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 28115
Aug 11 00:14:50 dist-test-slave-8hhm krb5kdc[26495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.3@KRBTEST.COM
I20260811 00:14:50.624452 28405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:50.606852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50368 (local address 127.7.94.3:33999)
0811 00:14:50.607081 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 00:14:50.607086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:14:50.607107 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:14:50.607346 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:14:50.607351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:14:50.607418 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:14:50.607542 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:14:50.609430 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.610298 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.612197 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:14:50.612641 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:14:50.612744 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:14:50.618246 (+ 5502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:14:50.618258 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:14:50.618276 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:14:50.618308 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:14:50.621551 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:14:50.622293 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:14:50.622301 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:14:50.622303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:14:50.622441 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:14:50.622851 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:14:50.622862 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:14:50.622865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:14:50.623195 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:14:50.623536 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 00:14:50.623821 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 00:14:50.624075 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 00:14:50.625370 28371 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:14:50.671214 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:50.670728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:43193 (local address 127.0.0.1:48520)
0811 00:14:50.670928 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 00:14:50.671025 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:43193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 00:14:50.672302 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:14:50.671913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:43193 (local address 127.0.0.1:48528)
0811 00:14:50.672122 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 00:14:50.672202 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:43193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 00:14:50.746579 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 28115
I20260811 00:14:50.777347 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 27348
I20260811 00:14:50.778563 27465 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:14:50.892092 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 27348
I20260811 00:14:50.931505 7544 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 00:14:51.278354 7544 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 00:14:51.278626 7544 mini_ranger.cc:66] Stopping Ranger...
I20260811 00:14:51.633692 7544 mini_ranger.cc:68] Stopped Ranger
2026-08-11 00:14:51.634 UTC [26569] LOG: received smart shutdown request
2026-08-11 00:14:51.638 UTC [26569] LOG: background worker "logical replication launcher" (PID 26594) exited with exit code 1
2026-08-11 00:14:51.638 UTC [26578] LOG: shutting down
2026-08-11 00:14:51.788 UTC [26569] LOG: database system is shut down
2026-08-11T00:14:51Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52216 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:14:52 dist-test-slave-8hhm krb5kdc[28487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:14:52 dist-test-slave-8hhm krb5kdc[28487](info): set up 2 sockets
Aug 11 00:14:52 dist-test-slave-8hhm krb5kdc[28487](info): commencing operation
krb5kdc: starting...
W20260811 00:14:53.284677 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.001s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:14:53 dist-test-slave-8hhm krb5kdc[28487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:14:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:14:53Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.94.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.94.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.94.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/rangeradmin_127.7.94.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.94.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/rangeradmin_127.7.94.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.94.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.94.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.94.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/rangerlookup_127.7.94.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.94.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/rangerlookup_127.7.94.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.94.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.94.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.94.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/HTTP_127.7.94.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.94.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/HTTP_127.7.94.20@KRBTEST.COM.keytab.
I20260811 00:14:53.657480 7544 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task1KEo_S/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/postgres -l logfile start
2026-08-11 00:14:56.067 UTC [28544] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 00:14:56.067 UTC [28544] LOG: listening on IPv4 address "127.7.94.20", port 60611
2026-08-11 00:14:56.071 UTC [28544] LOG: listening on Unix socket "/tmp/.s.PGSQL.60611"
2026-08-11 00:14:56.095 UTC [28548] LOG: database system was shut down at 2026-08-11 00:14:54 UTC
2026-08-11 00:14:56.098 UTC [28544] LOG: database system is ready to accept connections
I20260811 00:14:57.311784 7544 mini_postgres.cc:96] Postgres bound to 60611
2026-08-11 00:14:57.335 UTC [28561] FATAL: database "slave" does not exist
127.7.94.20:60611 - accepting connections
I20260811 00:14:57.335736 7544 mini_ranger.cc:161] Starting Ranger...
I20260811 00:14:57.366084 7544 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 00:14:57.846789 7544 mini_ranger.cc:87] Created ranger Postgres database
I20260811 00:14:57.846920 7544 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ranger-admin
2026-08-11 00:14:58,135 [I] DB FLAVOR :POSTGRES
2026-08-11 00:14:58,136 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 00:14:58,136 [I] Checking connection..
2026-08-11 00:14:58,136 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 00:14:58,388 [I] Checking connection passed.
2026-08-11 00:14:58,388 [I] --------- Verifying version history table ---------
2026-08-11 00:14:58,389 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 00:14:58,653 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 00:14:58,653 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 00:14:58,890 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 00:14:58,890 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 00:14:58,891 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 00:14:59.130 UTC [28687] WARNING: there is no transaction in progress
2026-08-11 00:14:59,139 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 00:14:59,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 00:14:59,405 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 00:14:59,405 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 00:14:59,405 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 00:14:59,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 00:14:59,917 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8hhm.c.gcp-upstream.internal','N') ;"
2026-08-11 00:15:00,182 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 00:15:00,447 [I] Table x_portal_user does not exist in database ranger
2026-08-11 00:15:00,447 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 00:15:00,723 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 00:15:00,723 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 00:15:00,988 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 00:15:01,233 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 00:15:01,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 00:15:02.166 UTC [28887] WARNING: there is no transaction in progress
2026-08-11 00:15:02.181 UTC [28887] WARNING: there is no transaction in progress
2026-08-11 00:15:02.197 UTC [28887] WARNING: there is no transaction in progress
2026-08-11 00:15:02.211 UTC [28887] WARNING: there is no transaction in progress
2026-08-11 00:15:02.227 UTC [28887] WARNING: there is no transaction in progress
2026-08-11 00:15:02.244 UTC [28887] WARNING: there is no transaction in progress
2026-08-11 00:15:02.297 UTC [28887] WARNING: there is no transaction in progress
2026-08-11 00:15:02.304 UTC [28887] WARNING: there is no transaction in progress
2026-08-11 00:15:02.319 UTC [28887] WARNING: there is no transaction in progress
2026-08-11 00:15:02.330 UTC [28887] WARNING: there is no transaction in progress
2026-08-11 00:15:02.340 UTC [28887] WARNING: there is no transaction in progress
2026-08-11 00:15:02.354 UTC [28887] WARNING: there is no transaction in progress
2026-08-11 00:15:02.364 UTC [28887] WARNING: there is no transaction in progress
2026-08-11 00:15:02.370 UTC [28887] WARNING: there is no transaction in progress
2026-08-11 00:15:02.376 UTC [28887] WARNING: there is no transaction in progress
2026-08-11 00:15:02.382 UTC [28887] WARNING: there is no transaction in progress
2026-08-11 00:15:02,988 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 00:15:02,989 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 00:15:03,236 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 00:15:03,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 00:15:03,479 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 00:15:03,479 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 00:15:03,752 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 00:15:03,753 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 00:15:04,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 00:15:04,271 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8hhm.c.gcp-upstream.internal';"
2026-08-11 00:15:04,537 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 00:15:04,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 00:15:04,786 [I] DB_PATCHES have already been applied
I20260811 00:15:04.792514 7544 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ranger-admin:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task1KEo_S/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 00:15:04.792610 7544 mini_ranger.cc:193] Using host: 127.7.94.20
I20260811 00:15:04.813081 7544 mini_ranger.cc:227] Ranger admin URL: http://127.7.94.20:36001
Aug 11, 2026 12:15:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:15:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:15:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 12:15:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 12:15:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 12:15:06 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 12:15:06 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 00:15:07.094072 7544 mini_ranger.cc:160] Time spent starting Ranger: real 9.758s user 0.006s sys 0.055s
Aug 11, 2026 12:15:26 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 12:15:26 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 12:15:26 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 12:15:26 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 12:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 12:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 12:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 12:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 12:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 12:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 12:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 12:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 12:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 12:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 12:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 12:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 12:15:26 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 12:15:27 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 00:15:27.794689 7544 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.94.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.94.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.94.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/rangerkms_127.7.94.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.94.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/rangerkms_127.7.94.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.94.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.94.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.94.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/HTTP_127.7.94.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.94.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/HTTP_127.7.94.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 00:15:27 dist-test-slave-8hhm krb5kdc[28487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407327, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 00:15:27.985929 7544 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 00:15:28.012326 7544 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 00:15:28.830036 7544 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 00:15:28.830188 7544 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ranger-kms
I20260811 00:15:28.830235 7544 mini_ranger_kms.cc:227] Using postgres at 127.7.94.20:60611
2026-08-11 00:15:29,195 [I] DB FLAVOR :POSTGRES
2026-08-11 00:15:29,195 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 00:15:29,195 [I] Checking connection
2026-08-11 00:15:29,196 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 00:15:29,442 [I] connection success
2026-08-11 00:15:29,443 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 00:15:29,443 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 00:15:29,443 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 00:15:29,686 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 00:15:29,687 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 00:15:29,687 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 00:15:29,687 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.94.20:60611/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 00:15:29,943 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 00:15:30.391741 7544 mini_ranger_kms.cc:310] Created kms service
I20260811 00:15:30.570331 7544 mini_ranger_kms.cc:326] Created kudu user
I20260811 00:15:30.613401 7544 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 00:15:30 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407327, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
I20260811 00:15:30.995640 7544 mini_ranger_kms.cc:384] Added ranger policy
I20260811 00:15:30.995960 7544 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ranger-kms:/tmp/dist-test-task1KEo_S/build/asan/bin/postgresql.jar:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task1KEo_S/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 00:15:30.995998 7544 mini_ranger_kms.cc:240] Using host: 127.7.94.20
I20260811 00:15:31.022693 7544 mini_ranger_kms.cc:276] Ranger KMS PID: 29216
I20260811 00:15:31.022835 7544 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.94.20:44479
Aug 11, 2026 12:15:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:15:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:15:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 12:15:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 12:15:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 12:15:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 00:15:33 dist-test-slave-8hhm krb5kdc[28487](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786407333, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.94.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 00:15:33 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407333, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.94.20@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
I20260811 00:15:33.961059 7544 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.975s user 0.064s sys 0.328s
I20260811 00:15:33.961341 7544 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 00:15:33.961524 7544 mini_ranger_kms.cc:401] 127.7.94.20:44479/kms/v1/keys
Aug 11 00:15:34 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407333, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.94.20@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
Aug 11 00:15:36 dist-test-slave-8hhm krb5kdc[28487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:15:36.561784 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:41469
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:41469
--ranger_config_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.94.20:44479/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:15:36.907749 29319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:15:36.908097 29319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:15:36.908241 29319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:15:36.915477 29319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:15:36.915583 29319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:15:36.915633 29319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:15:36.915683 29319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:15:36.915722 29319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:15:36.925915 29319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.94.20:44479/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:41469
--ranger_config_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:41469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:15:36.927256 29319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:15:36.928663 29319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:15:36.941049 29330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:36.941076 29333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:36.941133 29331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:36.941903 29319 server_base.cc:1034] running on GCE node
I20260811 00:15:36.942713 29319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:15:36.944109 29319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:15:36.945313 29319 hybrid_clock.cc:648] HybridClock initialized: now 1786407336945271 us; error 47 us; skew 500 ppm
I20260811 00:15:36.945667 29319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:15:36 dist-test-slave-8hhm krb5kdc[28487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407336, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:36.948768 29319 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:15:36.950289 29319 webserver.cc:460] Webserver started at http://127.7.94.62:36687/ using document root <none> and password file <none>
I20260811 00:15:36.950949 29319 fs_manager.cc:359] Metadata directory not provided
I20260811 00:15:36.951099 29319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:15:36.951411 29319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:15:36 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407336, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
I20260811 00:15:37.008760 29319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "eb40369aef934468b4d1e72fdbabec0e"
format_stamp: "Formatted at 2026-08-11 00:15:37 on dist-test-slave-8hhm"
server_key: "366461dddce9e47eaf6974ae8e23d025"
server_key_iv: "4b91e312b338ae7cc4db01e051989343"
server_key_version: "kuduclusterkey@0"
I20260811 00:15:37.009691 29319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "eb40369aef934468b4d1e72fdbabec0e"
format_stamp: "Formatted at 2026-08-11 00:15:37 on dist-test-slave-8hhm"
server_key: "366461dddce9e47eaf6974ae8e23d025"
server_key_iv: "4b91e312b338ae7cc4db01e051989343"
server_key_version: "kuduclusterkey@0"
I20260811 00:15:37.015185 29319 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:15:37.034515 29343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:37.036213 29319 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 00:15:37.036406 29319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "eb40369aef934468b4d1e72fdbabec0e"
format_stamp: "Formatted at 2026-08-11 00:15:37 on dist-test-slave-8hhm"
server_key: "366461dddce9e47eaf6974ae8e23d025"
server_key_iv: "4b91e312b338ae7cc4db01e051989343"
server_key_version: "kuduclusterkey@0"
I20260811 00:15:37.036667 29319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:15:37.065402 29319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:15:37.069165 29319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:15:37.069542 29319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:15:37.088160 29319 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:41469
I20260811 00:15:37.088195 29425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:41469 every 8 connection(s)
I20260811 00:15:37.089707 29319 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:15:37.091459 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 29319
I20260811 00:15:37.091715 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:15:37.092406 7544 external_mini_cluster.cc:1428] Setting key 1c4e4bf7f6c3ce5485435e84a409fa0f
I20260811 00:15:37.093803 29426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 00:15:37 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:15:37.107705 29426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e: Bootstrap starting.
I20260811 00:15:37.111894 29426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e: Neither blocks nor log segments found. Creating new log.
I20260811 00:15:37.113155 29426 log.cc:824] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e: Log is configured to *not* fsync() on all Append() calls
I20260811 00:15:37.116560 29426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e: No bootstrap required, opened a new log
I20260811 00:15:37.116780 29430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:37.094537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52664 (local address 127.7.94.62:41469)
0811 00:15:37.095417 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:37.095433 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:37.095484 (+ 51us) server_negotiation.cc:407] Connection header received
0811 00:15:37.097356 (+ 1872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:37.097386 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:37.097893 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:37.098342 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:37.099547 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:37.101556 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:37.102889 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:37.103905 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:37.104091 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:37.108301 (+ 4210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:37.108311 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:37.108329 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:37.108370 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:37.112471 (+ 4101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:37.113102 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:37.113112 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:37.113120 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:37.113347 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:37.113713 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:37.113722 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:37.113727 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:37.114366 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:37.114648 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:37.115083 (+ 435us) server_negotiation.cc:299] Negotiation successful
0811 00:15:37.115487 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":396,"threads_started":1}
I20260811 00:15:37.124708 29426 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb40369aef934468b4d1e72fdbabec0e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41469 } }
I20260811 00:15:37.125063 29426 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:15:37.125168 29426 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb40369aef934468b4d1e72fdbabec0e, State: Initialized, Role: FOLLOWER
I20260811 00:15:37.125773 29426 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb40369aef934468b4d1e72fdbabec0e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41469 } }
I20260811 00:15:37.125984 29426 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:15:37.126076 29426 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:15:37.126245 29426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:15:37.128527 29426 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb40369aef934468b4d1e72fdbabec0e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41469 } }
I20260811 00:15:37.128950 29426 leader_election.cc:302] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb40369aef934468b4d1e72fdbabec0e; no voters:
I20260811 00:15:37.129469 29426 leader_election.cc:288] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:15:37.129626 29433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:15:37.130352 29433 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [term 1 LEADER]: Becoming Leader. State: Replica: eb40369aef934468b4d1e72fdbabec0e, State: Running, Role: LEADER
I20260811 00:15:37.130859 29433 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb40369aef934468b4d1e72fdbabec0e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41469 } }
I20260811 00:15:37.131593 29426 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:15:37.141155 29435 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb40369aef934468b4d1e72fdbabec0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb40369aef934468b4d1e72fdbabec0e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41469 } } }
I20260811 00:15:37.141597 29435 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [sys.catalog]: This master's current role is: LEADER
I20260811 00:15:37.142098 29426 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 00:15:37.142405 29436 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb40369aef934468b4d1e72fdbabec0e. Latest consensus state: current_term: 1 leader_uuid: "eb40369aef934468b4d1e72fdbabec0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb40369aef934468b4d1e72fdbabec0e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41469 } } }
I20260811 00:15:37.142616 29436 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [sys.catalog]: This master's current role is: LEADER
I20260811 00:15:37.142951 29464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:15:37.154803 29464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:15:37.165807 29464 catalog_manager.cc:1347] Generated new cluster ID: 72a8e8f925894c018d0a5a6b75cf599a
I20260811 00:15:37.165948 29464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:15:37.182724 29464 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:15:37.184588 29464 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:15:37.194571 29464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e: Generated new TSK 0
I20260811 00:15:37.195677 29464 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
I20260811 00:15:37.981442 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.94.20:44479/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41469
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:15:38 dist-test-slave-8hhm krb5kdc[28487](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786407338, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 00:15:38.429492 29507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:15:38.429996 29507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:15:38.430218 29507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:15:38.442145 29507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:15:38.442308 29507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:15:38.442541 29507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:15:38.458261 29507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.94.20:44479/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:15:38.460421 29507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:15:38.462323 29507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:15:38.478415 29531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:38.478645 29530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:38.479282 29535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:38.479341 29507 server_base.cc:1034] running on GCE node
I20260811 00:15:38.480037 29507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:15:38.481010 29507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:15:38.482234 29507 hybrid_clock.cc:648] HybridClock initialized: now 1786407338482167 us; error 67 us; skew 500 ppm
I20260811 00:15:38.482555 29507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:15:38 dist-test-slave-8hhm krb5kdc[28487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407338, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:38.486440 29507 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:15:38.488476 29507 webserver.cc:460] Webserver started at http://127.7.94.1:41299/ using document root <none> and password file <none>
I20260811 00:15:38.489452 29507 fs_manager.cc:359] Metadata directory not provided
I20260811 00:15:38.489632 29507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:15:38.490046 29507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:15:38 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407338, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
I20260811 00:15:38.514045 29507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "bd0ad5916f9a417bbf48414e86678ab2"
format_stamp: "Formatted at 2026-08-11 00:15:38 on dist-test-slave-8hhm"
server_key: "d7177facae1571530f83561d3944b638"
server_key_iv: "80c0b8cec83179568e1130251fc516c0"
server_key_version: "kuduclusterkey@0"
I20260811 00:15:38.515969 29507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "bd0ad5916f9a417bbf48414e86678ab2"
format_stamp: "Formatted at 2026-08-11 00:15:38 on dist-test-slave-8hhm"
server_key: "d7177facae1571530f83561d3944b638"
server_key_iv: "80c0b8cec83179568e1130251fc516c0"
server_key_version: "kuduclusterkey@0"
I20260811 00:15:38.529520 29507 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.008s sys 0.005s
I20260811 00:15:38.570027 29547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:38.572322 29507 fs_manager.cc:725] Time spent opening block manager: real 0.024s user 0.002s sys 0.021s
I20260811 00:15:38.572655 29507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "bd0ad5916f9a417bbf48414e86678ab2"
format_stamp: "Formatted at 2026-08-11 00:15:38 on dist-test-slave-8hhm"
server_key: "d7177facae1571530f83561d3944b638"
server_key_iv: "80c0b8cec83179568e1130251fc516c0"
server_key_version: "kuduclusterkey@0"
I20260811 00:15:38.573035 29507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:15:38.602200 29507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:15:38.608111 29507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:15:38.608497 29507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:15:38.610741 29507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:15:38.614969 29507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:15:38.615164 29507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 00:15:38.615392 29507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:15:38.615541 29507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:15:38 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407338, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
I20260811 00:15:38.813650 29507 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:44221
I20260811 00:15:38.813657 29741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:44221 every 8 connection(s)
I20260811 00:15:38.815210 29507 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:15:38.821801 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 29507
I20260811 00:15:38.822055 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:15:38.822929 7544 external_mini_cluster.cc:1428] Setting key fd3d5586843f5b7925a97c37136e9c12
Aug 11 00:15:38 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407338, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:15:38.847496 29747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:38.819323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41469 (local address 127.7.94.1:53569)
0811 00:15:38.821912 (+ 2589us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:38.821993 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:38.824311 (+ 2318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:38.825144 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:38.825157 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:38.825847 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:38.826721 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:38.826752 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:38.828363 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:38.828370 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:38.830813 (+ 2443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:38.830832 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:38.832072 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:38.832081 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:38.832518 (+ 437us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:38.833482 (+ 964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:38.833517 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:38.837886 (+ 4369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:38.842390 (+ 4504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:38.842402 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:38.842423 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:38.843050 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:38.843555 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:38.843560 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:38.843564 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:38.843818 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:38.844582 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:38.844591 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:38.845023 (+ 432us) client_negotiation.cc:769] Sending connection context
0811 00:15:38.845347 (+ 324us) client_negotiation.cc:240] Negotiation successful
0811 00:15:38.845942 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2239,"thread_start_us":2002,"threads_started":1}
I20260811 00:15:38.848685 29746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:38.818643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:53569 (local address 127.7.94.62:41469)
0811 00:15:38.822665 (+ 4022us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:38.822672 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:38.822698 (+ 26us) server_negotiation.cc:407] Connection header received
0811 00:15:38.824587 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:38.824593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:38.824711 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:38.824839 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:38.826951 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:38.828131 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:38.831020 (+ 2889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:38.831784 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:38.831934 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:38.838208 (+ 6274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:38.838215 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:38.838219 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:38.838261 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:38.842212 (+ 3951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:38.843222 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:38.843227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:38.843232 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:38.843397 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:38.843961 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:38.843966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:38.843969 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:38.844309 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:38.844502 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:38.848056 (+ 3554us) server_negotiation.cc:299] Negotiation successful
0811 00:15:38.848284 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3898,"thread_start_us":4165,"threads_started":1}
I20260811 00:15:38.850337 29744 heartbeater.cc:347] Connected to a master server at 127.7.94.62:41469
I20260811 00:15:38.850672 29744 heartbeater.cc:464] Registering TS with master...
I20260811 00:15:38.851429 29744 heartbeater.cc:511] Master 127.7.94.62:41469 requested a full tablet report, sending...
I20260811 00:15:38.854063 29376 ts_manager.cc:194] Registered new tserver with Master: bd0ad5916f9a417bbf48414e86678ab2 (127.7.94.1:44221)
I20260811 00:15:38.856186 29376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:53569
WARNING: no policy specified for kudu/127.7.94.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
I20260811 00:15:38.901507 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.94.20:44479/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41469
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:15:38 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407338, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
W20260811 00:15:39.370820 29752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:15:39.371157 29752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:15:39.371330 29752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:15:39.378577 29752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:15:39.378715 29752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:15:39.378868 29752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:15:39.388573 29752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.94.20:44479/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:15:39.390044 29752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:15:39.391316 29752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:15:39.404657 29780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:39.404779 29778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:39.405339 29782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:39.405735 29752 server_base.cc:1034] running on GCE node
I20260811 00:15:39.406401 29752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:15:39.407437 29752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:15:39.408784 29752 hybrid_clock.cc:648] HybridClock initialized: now 1786407339408734 us; error 36 us; skew 500 ppm
I20260811 00:15:39.409170 29752 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:15:39 dist-test-slave-8hhm krb5kdc[28487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407339, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:39.412510 29752 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:15:39.414245 29752 webserver.cc:460] Webserver started at http://127.7.94.2:37393/ using document root <none> and password file <none>
I20260811 00:15:39.415009 29752 fs_manager.cc:359] Metadata directory not provided
I20260811 00:15:39.415167 29752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:15:39.415490 29752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:15:39 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407339, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
I20260811 00:15:39.434368 29752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "721ccc5670564e3ca67a18562fa48bfd"
format_stamp: "Formatted at 2026-08-11 00:15:39 on dist-test-slave-8hhm"
server_key: "35fb01c9da2c62a4a3b26e29d03d7892"
server_key_iv: "5066cf8ae612a551b2b4179b132675ab"
server_key_version: "kuduclusterkey@0"
I20260811 00:15:39.435686 29752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "721ccc5670564e3ca67a18562fa48bfd"
format_stamp: "Formatted at 2026-08-11 00:15:39 on dist-test-slave-8hhm"
server_key: "35fb01c9da2c62a4a3b26e29d03d7892"
server_key_iv: "5066cf8ae612a551b2b4179b132675ab"
server_key_version: "kuduclusterkey@0"
I20260811 00:15:39.443120 29752 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.004s
I20260811 00:15:39.459498 29796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:39.462564 29752 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 00:15:39.462859 29752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "721ccc5670564e3ca67a18562fa48bfd"
format_stamp: "Formatted at 2026-08-11 00:15:39 on dist-test-slave-8hhm"
server_key: "35fb01c9da2c62a4a3b26e29d03d7892"
server_key_iv: "5066cf8ae612a551b2b4179b132675ab"
server_key_version: "kuduclusterkey@0"
I20260811 00:15:39.463205 29752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:15:39.480515 29752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:15:39.484509 29752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:15:39.484809 29752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:15:39.485913 29752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:15:39.487341 29752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:15:39.487444 29752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:39.487599 29752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:15:39.487673 29752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:39.527598 29752 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:37945
I20260811 00:15:39.527601 30002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:37945 every 8 connection(s)
I20260811 00:15:39.528952 29752 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:15:39.532418 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 29752
I20260811 00:15:39.532591 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:15:39.533453 7544 external_mini_cluster.cc:1428] Setting key 1fd12be3f006488e89984403fa1752b8
Aug 11 00:15:39 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407339, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:15:39.565385 30009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:39.531925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:33545 (local address 127.7.94.62:41469)
0811 00:15:39.547493 (+ 15568us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:39.547499 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:39.547521 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:15:39.547587 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:39.547593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:39.547673 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:39.547794 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:39.549881 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:39.550624 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:39.552248 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:39.552976 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:39.553120 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:39.557513 (+ 4393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:39.557521 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:39.557528 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:39.557571 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:39.561455 (+ 3884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:39.562401 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:39.562410 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:39.562416 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:39.562572 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:39.563107 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:39.563114 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:39.563120 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:39.563500 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:39.563649 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:39.564577 (+ 928us) server_negotiation.cc:299] Negotiation successful
0811 00:15:39.564842 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15425,"thread_start_us":15974,"threads_started":1}
I20260811 00:15:39.566514 30012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:39.532569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41469 (local address 127.7.94.2:33545)
0811 00:15:39.541712 (+ 9143us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:39.541774 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:39.544591 (+ 2817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:39.548130 (+ 3539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:39.548146 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:39.548826 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:39.549688 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:39.549710 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:39.550805 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:39.550811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:39.552072 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:39.552091 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:39.553298 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:39.553318 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:39.553641 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:39.554442 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:39.554480 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:39.557255 (+ 2775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:39.561631 (+ 4376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:39.561640 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:39.561657 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:39.562253 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:39.562695 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:39.562700 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:39.562704 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:39.562957 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:39.563645 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:39.563653 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:39.564088 (+ 435us) client_negotiation.cc:769] Sending connection context
0811 00:15:39.564387 (+ 299us) client_negotiation.cc:240] Negotiation successful
0811 00:15:39.564924 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8783,"thread_start_us":12343,"threads_started":1}
I20260811 00:15:39.568248 30004 heartbeater.cc:347] Connected to a master server at 127.7.94.62:41469
I20260811 00:15:39.568612 30004 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.94.3@KRBTEST.COM; defaulting to no policy
I20260811 00:15:39.569561 30004 heartbeater.cc:511] Master 127.7.94.62:41469 requested a full tablet report, sending...
I20260811 00:15:39.571172 29376 ts_manager.cc:194] Registered new tserver with Master: 721ccc5670564e3ca67a18562fa48bfd (127.7.94.2:37945)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.3@KRBTEST.COM" created.
I20260811 00:15:39.572116 29376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:33545
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
I20260811 00:15:39.604977 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.94.20:44479/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41469
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:15:39.859808 29744 heartbeater.cc:503] Master 127.7.94.62:41469 was elected leader, sending a full tablet report...
W20260811 00:15:39.946154 30015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:15:39.946489 30015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:15:39.946668 30015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:15:39.954114 30015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:15:39.954252 30015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:15:39.954428 30015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:15:39.964311 30015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.94.20:44479/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:15:39.965725 30015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:15:39.967043 30015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:15:39.980631 30025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:39.980705 30026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:39.980962 30036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:39.982169 30015 server_base.cc:1034] running on GCE node
I20260811 00:15:39.982861 30015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:15:39.983881 30015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:15:39.985131 30015 hybrid_clock.cc:648] HybridClock initialized: now 1786407339985061 us; error 69 us; skew 500 ppm
I20260811 00:15:39.985498 30015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:15:39 dist-test-slave-8hhm krb5kdc[28487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407339, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:39.988660 30015 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:15:39.990126 30015 webserver.cc:460] Webserver started at http://127.7.94.3:34463/ using document root <none> and password file <none>
I20260811 00:15:39.990856 30015 fs_manager.cc:359] Metadata directory not provided
I20260811 00:15:39.990991 30015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:15:39.991305 30015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 00:15:39 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407339, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
I20260811 00:15:40.008253 30015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "5f312aea29ea41cd94e0c4c60e245975"
format_stamp: "Formatted at 2026-08-11 00:15:39 on dist-test-slave-8hhm"
server_key: "e663350dc6874c7501a058f07f867c7e"
server_key_iv: "5bde6678e20a0ad9c84cfcbc0440bb19"
server_key_version: "kuduclusterkey@0"
I20260811 00:15:40.009147 30015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "5f312aea29ea41cd94e0c4c60e245975"
format_stamp: "Formatted at 2026-08-11 00:15:39 on dist-test-slave-8hhm"
server_key: "e663350dc6874c7501a058f07f867c7e"
server_key_iv: "5bde6678e20a0ad9c84cfcbc0440bb19"
server_key_version: "kuduclusterkey@0"
I20260811 00:15:40.014549 30015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 00:15:40.029019 30052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:40.030725 30015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:15:40.030941 30015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "5f312aea29ea41cd94e0c4c60e245975"
format_stamp: "Formatted at 2026-08-11 00:15:39 on dist-test-slave-8hhm"
server_key: "e663350dc6874c7501a058f07f867c7e"
server_key_iv: "5bde6678e20a0ad9c84cfcbc0440bb19"
server_key_version: "kuduclusterkey@0"
I20260811 00:15:40.031229 30015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:15:40.060350 30015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:15:40.064656 30015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:15:40.064967 30015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:15:40.066079 30015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:15:40.067471 30015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:15:40.067576 30015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:40.067720 30015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:15:40.067795 30015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:40.101447 30015 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:43811
I20260811 00:15:40.101466 30240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:43811 every 8 connection(s)
I20260811 00:15:40.102753 30015 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:15:40.104087 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 30015
I20260811 00:15:40.104293 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:15:40.104797 7544 external_mini_cluster.cc:1428] Setting key cc491f27ecad665f2b8a72da55ac5654
Aug 11 00:15:40 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407339, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:15:40.123696 30245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:40.105383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:33101 (local address 127.7.94.62:41469)
0811 00:15:40.105906 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:40.105911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:40.107175 (+ 1264us) server_negotiation.cc:407] Connection header received
0811 00:15:40.109274 (+ 2099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:40.109278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:40.109344 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:40.109431 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:40.110837 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:40.111567 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:40.113120 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:40.113628 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:40.113736 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:40.117163 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:40.117168 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:40.117171 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:40.117259 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:40.120492 (+ 3233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:40.121268 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:40.121275 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:40.121281 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:40.121420 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:40.121896 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:40.121901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:40.121904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:40.122224 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:40.122376 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:40.123094 (+ 718us) server_negotiation.cc:299] Negotiation successful
0811 00:15:40.123322 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":243,"threads_started":1}
I20260811 00:15:40.124744 30246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:40.106044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41469 (local address 127.7.94.3:33101)
0811 00:15:40.106993 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:40.107037 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:40.108985 (+ 1948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:40.109631 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:40.109639 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:40.110084 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:40.110687 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:40.110705 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:40.111713 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:40.111717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:40.112967 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:40.112987 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:40.113869 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:40.113874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:40.114082 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:40.114727 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:40.114749 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:40.116930 (+ 2181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:40.120647 (+ 3717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:40.120673 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:40.120697 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:40.121131 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:40.121554 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:40.121565 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:40.121567 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:40.121780 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:40.122376 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:40.122384 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:40.122725 (+ 341us) client_negotiation.cc:769] Sending connection context
0811 00:15:40.123003 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 00:15:40.123550 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":373,"threads_started":1}
I20260811 00:15:40.126137 30243 heartbeater.cc:347] Connected to a master server at 127.7.94.62:41469
I20260811 00:15:40.126477 30243 heartbeater.cc:464] Registering TS with master...
I20260811 00:15:40.127177 30243 heartbeater.cc:511] Master 127.7.94.62:41469 requested a full tablet report, sending...
I20260811 00:15:40.128538 29376 ts_manager.cc:194] Registered new tserver with Master: 5f312aea29ea41cd94e0c4c60e245975 (127.7.94.3:43811)
I20260811 00:15:40.129335 29376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:33101
I20260811 00:15:40.133121 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:15:40.134824 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 29507
Aug 11 00:15:40 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:15:40.151266 30251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:40.135798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47096 (local address 127.7.94.1:44221)
0811 00:15:40.136416 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:40.136422 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:40.136447 (+ 25us) server_negotiation.cc:407] Connection header received
0811 00:15:40.136531 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:40.136537 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:40.136708 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:40.136829 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:40.138333 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:40.139328 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:40.140916 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:40.141576 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:40.141695 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:40.145976 (+ 4281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:40.145986 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:40.146000 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:40.146031 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:40.148642 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:40.149342 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:40.149348 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:40.149351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:40.149480 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:40.149862 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:40.149865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:40.149867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:40.150189 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:40.150405 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:40.150694 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 00:15:40.150915 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":291,"threads_started":1}
I20260811 00:15:40.152585 29731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:15:40.235654 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 29507
I20260811 00:15:40.262342 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 29752
Aug 11 00:15:40 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.2@KRBTEST.COM
I20260811 00:15:40.285378 30255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:40.263281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58606 (local address 127.7.94.2:37945)
0811 00:15:40.263828 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:40.263836 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:40.263859 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:15:40.263924 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:40.263930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:40.264139 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:40.264313 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:40.265611 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:40.266615 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:40.268002 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:40.268549 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:40.268697 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:40.279353 (+ 10656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:40.279363 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:40.279395 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:40.279429 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:40.282277 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:40.283041 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:40.283049 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:40.283051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:40.283183 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:40.283572 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:40.283577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:40.283580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:40.283936 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:40.284259 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:40.284559 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 00:15:40.284917 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":297,"threads_started":1}
I20260811 00:15:40.286696 29995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:15:40.368603 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 29752
I20260811 00:15:40.399454 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 30015
Aug 11 00:15:40 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.3@KRBTEST.COM
I20260811 00:15:40.415844 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:40.400506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38326 (local address 127.7.94.3:43811)
0811 00:15:40.401133 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:40.401140 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:40.401171 (+ 31us) server_negotiation.cc:407] Connection header received
0811 00:15:40.401285 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:40.401292 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:40.401514 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:40.401684 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:40.403017 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:40.403987 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:40.405345 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:40.405903 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:40.406036 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:40.410168 (+ 4132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:40.410179 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:40.410194 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:40.410229 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:40.412787 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:40.413604 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:40.413611 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:40.413613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:40.413749 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:40.414193 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:40.414199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:40.414203 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:40.414571 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:40.414909 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:40.415213 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 00:15:40.415482 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":320,"threads_started":1}
I20260811 00:15:40.417117 30235 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:15:40.501915 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 30015
I20260811 00:15:40.529394 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 29319
I20260811 00:15:40.530656 29420 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:15:40.636831 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 29319
I20260811 00:15:40.672216 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:41469
--webserver_interface=127.7.94.62
--webserver_port=36687
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:41469
--ranger_config_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.94.20:44479/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.94.20:44479/kms with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:15:41.004297 30265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:15:41.004570 30265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:15:41.004657 30265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:15:41.011844 30265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:15:41.011958 30265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:15:41.012010 30265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:15:41.012070 30265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:15:41.012118 30265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:15:41.021907 30265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.94.20:44479/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:41469
--ranger_config_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:41469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=36687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:15:41.023133 30265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:15:41.024466 30265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:15:41.036067 30274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:41.036142 30278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:41.036113 30276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:41.036835 30265 server_base.cc:1034] running on GCE node
I20260811 00:15:41.037674 30265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:15:41.039021 30265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:15:41.040220 30265 hybrid_clock.cc:648] HybridClock initialized: now 1786407341040188 us; error 45 us; skew 500 ppm
I20260811 00:15:41.040567 30265 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:15:41 dist-test-slave-8hhm krb5kdc[28487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407341, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:41.043664 30265 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:15:41.045109 30265 webserver.cc:460] Webserver started at http://127.7.94.62:36687/ using document root <none> and password file <none>
I20260811 00:15:41.045773 30265 fs_manager.cc:359] Metadata directory not provided
I20260811 00:15:41.045883 30265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 00:15:41 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407341, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
I20260811 00:15:41.097520 30265 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 00:15:41.100790 30292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:41.102308 30265 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:15:41.102520 30265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "eb40369aef934468b4d1e72fdbabec0e"
format_stamp: "Formatted at 2026-08-11 00:15:37 on dist-test-slave-8hhm"
server_key: "366461dddce9e47eaf6974ae8e23d025"
server_key_iv: "4b91e312b338ae7cc4db01e051989343"
server_key_version: "kuduclusterkey@0"
I20260811 00:15:41.103546 30265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:15:41.132092 30265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:15:41.135974 30265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:15:41.136324 30265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:15:41.154940 30265 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:41469
I20260811 00:15:41.154961 30382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:41469 every 8 connection(s)
I20260811 00:15:41.156432 30265 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:15:41.159884 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 30265
I20260811 00:15:41.160732 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.94.20:44479/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:44221
--local_ip_for_outbound_sockets=127.7.94.1
--tserver_master_addrs=127.7.94.62:41469
--webserver_port=41299
--webserver_interface=127.7.94.1
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.94.20:44479/kms with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:15:41.163205 30383 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:15:41.166410 30383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e: Bootstrap starting.
I20260811 00:15:41.190354 30383 log.cc:824] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e: Log is configured to *not* fsync() on all Append() calls
I20260811 00:15:41.201875 30383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:15:41.202560 30383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e: Bootstrap complete.
I20260811 00:15:41.215199 30383 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb40369aef934468b4d1e72fdbabec0e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41469 } }
I20260811 00:15:41.216061 30383 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb40369aef934468b4d1e72fdbabec0e, State: Initialized, Role: FOLLOWER
I20260811 00:15:41.216773 30383 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb40369aef934468b4d1e72fdbabec0e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41469 } }
I20260811 00:15:41.217013 30383 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:15:41.217131 30383 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:15:41.217360 30383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [term 1 FOLLOWER]: Advancing to term 2
I20260811 00:15:41.221057 30383 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb40369aef934468b4d1e72fdbabec0e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41469 } }
I20260811 00:15:41.221740 30383 leader_election.cc:302] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb40369aef934468b4d1e72fdbabec0e; no voters:
I20260811 00:15:41.222533 30383 leader_election.cc:288] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 00:15:41.222672 30389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [term 2 FOLLOWER]: Leader election won for term 2
I20260811 00:15:41.223822 30389 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [term 2 LEADER]: Becoming Leader. State: Replica: eb40369aef934468b4d1e72fdbabec0e, State: Running, Role: LEADER
I20260811 00:15:41.224396 30389 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb40369aef934468b4d1e72fdbabec0e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41469 } }
I20260811 00:15:41.224833 30383 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:15:41.227157 30391 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb40369aef934468b4d1e72fdbabec0e. Latest consensus state: current_term: 2 leader_uuid: "eb40369aef934468b4d1e72fdbabec0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb40369aef934468b4d1e72fdbabec0e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41469 } } }
I20260811 00:15:41.227638 30391 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [sys.catalog]: This master's current role is: LEADER
I20260811 00:15:41.228195 30390 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "eb40369aef934468b4d1e72fdbabec0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb40369aef934468b4d1e72fdbabec0e" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41469 } } }
I20260811 00:15:41.228513 30390 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e [sys.catalog]: This master's current role is: LEADER
I20260811 00:15:41.238296 30397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:15:41.245178 30397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:15:41.246939 30397 catalog_manager.cc:1259] Loaded cluster ID: 72a8e8f925894c018d0a5a6b75cf599a
I20260811 00:15:41.247068 30397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:15:41.252279 30397 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:15:41.254177 30397 catalog_manager.cc:5879] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e: Loaded TSK: 0
I20260811 00:15:41.256058 30397 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:15:41.262607 30383 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260811 00:15:41.656894 30387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:15:41.657325 30387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:15:41.657492 30387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:15:41.668475 30387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:15:41.668634 30387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:15:41.668838 30387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:15:41.684777 30387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.94.20:44479/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:44221
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=41299
--tserver_master_addrs=127.7.94.62:41469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:15:41.686667 30387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:15:41.688485 30387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:15:41.711743 30451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:41.713018 30452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:41.721088 30455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:41.721678 30387 server_base.cc:1034] running on GCE node
I20260811 00:15:41.722438 30387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:15:41.724826 30387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:15:41.726138 30387 hybrid_clock.cc:648] HybridClock initialized: now 1786407341725959 us; error 116 us; skew 500 ppm
I20260811 00:15:41.726590 30387 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:15:41 dist-test-slave-8hhm krb5kdc[28487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407341, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:41.730623 30387 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:15:41.732596 30387 webserver.cc:460] Webserver started at http://127.7.94.1:41299/ using document root <none> and password file <none>
I20260811 00:15:41.733433 30387 fs_manager.cc:359] Metadata directory not provided
I20260811 00:15:41.733605 30387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 00:15:41 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407341, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
I20260811 00:15:41.761826 30387 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 00:15:41.765733 30469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:41.767499 30387 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 00:15:41.767711 30387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "bd0ad5916f9a417bbf48414e86678ab2"
format_stamp: "Formatted at 2026-08-11 00:15:38 on dist-test-slave-8hhm"
server_key: "d7177facae1571530f83561d3944b638"
server_key_iv: "80c0b8cec83179568e1130251fc516c0"
server_key_version: "kuduclusterkey@0"
I20260811 00:15:41.768800 30387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:15:41.788085 30387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:15:41.792748 30387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:15:41.793109 30387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:15:41.795323 30387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:15:41.797297 30387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:15:41.797430 30387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:41.797556 30387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:15:41.797646 30387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:41.865912 30387 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:44221
I20260811 00:15:41.865972 30663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:44221 every 8 connection(s)
I20260811 00:15:41.867697 30387 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:15:41.870069 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 30387
I20260811 00:15:41.870867 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.94.20:44479/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:37945
--local_ip_for_outbound_sockets=127.7.94.2
--tserver_master_addrs=127.7.94.62:41469
--webserver_port=37393
--webserver_interface=127.7.94.2
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.94.20:44479/kms with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:15:41 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407341, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:15:41.925168 30666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:41.882834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41469 (local address 127.7.94.1:35719)
0811 00:15:41.896278 (+ 13444us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:41.896331 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:41.898915 (+ 2584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:41.901342 (+ 2427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:41.901355 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:41.902042 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:41.902930 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:41.902959 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:41.905394 (+ 2435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:41.905402 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:41.907630 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:41.907649 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:41.909532 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:41.909544 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:41.909891 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:41.910804 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:41.910845 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:41.914024 (+ 3179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:41.919210 (+ 5186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:41.919222 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:41.919243 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:41.919933 (+ 690us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:41.920616 (+ 683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:41.920622 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:41.920628 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:41.920941 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:41.922286 (+ 1345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:41.922297 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:41.922751 (+ 454us) client_negotiation.cc:769] Sending connection context
0811 00:15:41.923199 (+ 448us) client_negotiation.cc:240] Negotiation successful
0811 00:15:41.923778 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11243,"thread_start_us":12300,"threads_started":1}
I20260811 00:15:41.925164 30668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:41.882828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:35719 (local address 127.7.94.62:41469)
0811 00:15:41.896697 (+ 13869us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:41.896722 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:41.896790 (+ 68us) server_negotiation.cc:407] Connection header received
0811 00:15:41.899594 (+ 2804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:41.899616 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:41.900149 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:41.900556 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:41.903206 (+ 2650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:41.905170 (+ 1964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:41.907806 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:41.908885 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:41.909080 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:41.914371 (+ 5291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:41.914385 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:41.914410 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:41.914465 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:41.918998 (+ 4533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:41.920199 (+ 1201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:41.920212 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:41.920223 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:41.920448 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:41.921112 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:41.921120 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:41.921126 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:41.921942 (+ 816us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:41.922197 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:41.923413 (+ 1216us) server_negotiation.cc:299] Negotiation successful
0811 00:15:41.923858 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13498,"thread_start_us":13801,"threads_started":1}
I20260811 00:15:41.929561 30664 heartbeater.cc:347] Connected to a master server at 127.7.94.62:41469
I20260811 00:15:41.929934 30664 heartbeater.cc:464] Registering TS with master...
I20260811 00:15:41.930784 30664 heartbeater.cc:511] Master 127.7.94.62:41469 requested a full tablet report, sending...
W20260811 00:15:41.934312 30664 heartbeater.cc:644] Failed to heartbeat to 127.7.94.62:41469 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 00:15:41.934558 30664 heartbeater.cc:464] Registering TS with master...
I20260811 00:15:41.934851 30664 heartbeater.cc:511] Master 127.7.94.62:41469 requested a full tablet report, sending...
I20260811 00:15:41.936093 30664 heartbeater.cc:464] Registering TS with master...
I20260811 00:15:41.936349 30664 heartbeater.cc:511] Master 127.7.94.62:41469 requested a full tablet report, sending...
W20260811 00:15:41.937528 30664 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 00:15:42.301911 30669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:15:42.302204 30669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:15:42.302371 30669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:15:42.309888 30669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:15:42.309991 30669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:15:42.310151 30669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:15:42.323319 30669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.94.20:44479/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:37945
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=37393
--tserver_master_addrs=127.7.94.62:41469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:15:42.325042 30669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:15:42.326772 30669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:15:42.343165 30688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:42.344872 30693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:42.346720 30696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:42.347687 30669 server_base.cc:1034] running on GCE node
I20260811 00:15:42.348311 30669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:15:42.349383 30669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:15:42.350670 30669 hybrid_clock.cc:648] HybridClock initialized: now 1786407342350604 us; error 51 us; skew 500 ppm
I20260811 00:15:42.351087 30669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:15:42 dist-test-slave-8hhm krb5kdc[28487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:42.355705 30669 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:15:42.361615 30669 webserver.cc:460] Webserver started at http://127.7.94.2:37393/ using document root <none> and password file <none>
I20260811 00:15:42.362572 30669 fs_manager.cc:359] Metadata directory not provided
I20260811 00:15:42.362742 30669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 00:15:42 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
I20260811 00:15:42.408399 30669 fs_manager.cc:709] Time spent opening directory manager: real 0.016s user 0.000s sys 0.015s
I20260811 00:15:42.412431 30711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:42.414477 30669 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:15:42.414732 30669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "721ccc5670564e3ca67a18562fa48bfd"
format_stamp: "Formatted at 2026-08-11 00:15:39 on dist-test-slave-8hhm"
server_key: "35fb01c9da2c62a4a3b26e29d03d7892"
server_key_iv: "5066cf8ae612a551b2b4179b132675ab"
server_key_version: "kuduclusterkey@0"
I20260811 00:15:42.416260 30669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:15:42.479425 30669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:15:42.485590 30669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:15:42.485944 30669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:15:42.487361 30669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:15:42.489449 30669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:15:42.489584 30669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:42.489702 30669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:15:42.489820 30669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 00:15:42 dist-test-slave-8hhm krb5kdc[28487](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786407342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:42.578363 30669 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:37945
I20260811 00:15:42.578415 30902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:37945 every 8 connection(s)
I20260811 00:15:42.579908 30669 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:15:42.588634 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 30669
I20260811 00:15:42.589512 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.94.20:44479/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:43811
--local_ip_for_outbound_sockets=127.7.94.3
--tserver_master_addrs=127.7.94.62:41469
--webserver_port=34463
--webserver_interface=127.7.94.3
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.94.20:44479/kms with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:15:42 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:15:42.607770 30907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:42.583285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:36313 (local address 127.7.94.62:41469)
0811 00:15:42.583833 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:42.583840 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:42.584869 (+ 1029us) server_negotiation.cc:407] Connection header received
0811 00:15:42.586915 (+ 2046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:42.586921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:42.587017 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:42.587148 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:42.589059 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:42.590272 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:42.593184 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:42.593994 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:42.594111 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:42.598887 (+ 4776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:42.598896 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:42.598903 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:42.598948 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:42.604155 (+ 5207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:42.604933 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:42.604939 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:42.604943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:42.605089 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:42.605635 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:42.605644 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:42.605647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:42.605986 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:42.606120 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:42.607138 (+ 1018us) server_negotiation.cc:299] Negotiation successful
0811 00:15:42.607359 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":275,"threads_started":1}
I20260811 00:15:42.608975 30910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:42.583841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41469 (local address 127.7.94.2:36313)
0811 00:15:42.584705 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:42.584760 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:42.586630 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:42.587410 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:42.587419 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:42.587917 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:42.588836 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:42.588864 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:42.590756 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:42.590764 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:42.593007 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:42.593026 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:42.594514 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:42.594523 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:42.594852 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:42.595674 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:42.595707 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:42.598633 (+ 2926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:42.604316 (+ 5683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:42.604325 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:42.604344 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:42.604797 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:42.605245 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:42.605250 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:42.605254 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:42.605504 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:42.606209 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:42.606217 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:42.606659 (+ 442us) client_negotiation.cc:769] Sending connection context
0811 00:15:42.606958 (+ 299us) client_negotiation.cc:240] Negotiation successful
0811 00:15:42.607536 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":346,"threads_started":1}
I20260811 00:15:42.610726 30903 heartbeater.cc:347] Connected to a master server at 127.7.94.62:41469
I20260811 00:15:42.611073 30903 heartbeater.cc:464] Registering TS with master...
I20260811 00:15:42.611824 30903 heartbeater.cc:511] Master 127.7.94.62:41469 requested a full tablet report, sending...
I20260811 00:15:42.613692 30326 ts_manager.cc:194] Registered new tserver with Master: 721ccc5670564e3ca67a18562fa48bfd (127.7.94.2:37945)
I20260811 00:15:42.615351 30326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:36313
I20260811 00:15:42.937671 30664 heartbeater.cc:464] Registering TS with master...
I20260811 00:15:42.938119 30664 heartbeater.cc:511] Master 127.7.94.62:41469 requested a full tablet report, sending...
I20260811 00:15:42.939404 30326 ts_manager.cc:194] Registered new tserver with Master: bd0ad5916f9a417bbf48414e86678ab2 (127.7.94.1:44221)
I20260811 00:15:42.940539 30326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:35719
Aug 11 00:15:42 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
W20260811 00:15:43.082485 30912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:15:43.082923 30912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:15:43.083091 30912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:15:43.094825 30912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:15:43.094978 30912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:15:43.095168 30912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:15:43.110177 30912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.94.20:44479/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:43811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=34463
--tserver_master_addrs=127.7.94.62:41469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:15:43.111598 30912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:15:43.113482 30912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:15:43.130573 30927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:43.131695 30933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:43.136580 30912 server_base.cc:1034] running on GCE node
W20260811 00:15:43.137260 30935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:43.138517 30912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:15:43.139808 30912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:15:43.141060 30912 hybrid_clock.cc:648] HybridClock initialized: now 1786407343140991 us; error 51 us; skew 500 ppm
I20260811 00:15:43.141464 30912 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:15:43 dist-test-slave-8hhm krb5kdc[28487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407343, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:43.145867 30912 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:15:43.147818 30912 webserver.cc:460] Webserver started at http://127.7.94.3:34463/ using document root <none> and password file <none>
I20260811 00:15:43.148696 30912 fs_manager.cc:359] Metadata directory not provided
I20260811 00:15:43.148833 30912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 00:15:43 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407343, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
I20260811 00:15:43.183316 30912 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.006s sys 0.006s
I20260811 00:15:43.187474 30951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:43.189498 30912 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 00:15:43.189673 30912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "5f312aea29ea41cd94e0c4c60e245975"
format_stamp: "Formatted at 2026-08-11 00:15:39 on dist-test-slave-8hhm"
server_key: "e663350dc6874c7501a058f07f867c7e"
server_key_iv: "5bde6678e20a0ad9c84cfcbc0440bb19"
server_key_version: "kuduclusterkey@0"
I20260811 00:15:43.191110 30912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:15:43.239300 30912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:15:43.244866 30912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:15:43.245271 30912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:15:43.247184 30912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:15:43.248912 30912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:15:43.249048 30912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:43.249194 30912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:15:43.249344 30912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 11 00:15:43 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407342, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for HTTP/127.7.94.20@KRBTEST.COM
I20260811 00:15:43.620031 30903 heartbeater.cc:503] Master 127.7.94.62:41469 was elected leader, sending a full tablet report...
I20260811 00:15:43.661567 30912 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:43811
I20260811 00:15:43.661619 31134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:43811 every 8 connection(s)
I20260811 00:15:43.663597 30912 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:15:43.674038 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 30912
Aug 11 00:15:43 dist-test-slave-8hhm krb5kdc[28487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407343, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:15:43.707808 31153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:43.675085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37400 (local address 127.7.94.62:41469)
0811 00:15:43.691112 (+ 16027us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:43.691119 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:43.691142 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:15:43.691230 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:43.691237 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:43.691368 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:43.691531 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:43.693742 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.694814 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.696693 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.697440 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.697594 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:43.700936 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:43.700941 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:43.700947 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:43.700978 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:43.704858 (+ 3880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:43.705755 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:43.705765 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:43.705768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:43.705902 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:43.706334 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:43.706337 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:43.706339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:43.706695 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:43.706940 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:43.707316 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 00:15:43.707474 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":15905,"spinlock_wait_cycles":1920,"thread_start_us":15371,"threads_started":1}
I20260811 00:15:43.713616 31154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:43.671256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:34159 (local address 127.7.94.62:41469)
0811 00:15:43.690718 (+ 19462us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:43.690727 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:43.692599 (+ 1872us) server_negotiation.cc:407] Connection header received
0811 00:15:43.695080 (+ 2481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:43.695085 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:43.695177 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:43.695302 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:43.697152 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.698441 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.700849 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.701608 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.701729 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:43.705819 (+ 4090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:43.705826 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:43.705831 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:43.705881 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:43.710445 (+ 4564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:43.711123 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:43.711128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:43.711131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:43.711274 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:43.711725 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:43.711730 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:43.711733 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:43.712082 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:43.712257 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:43.712957 (+ 700us) server_negotiation.cc:299] Negotiation successful
0811 00:15:43.713189 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":19298,"thread_start_us":19598,"threads_started":1}
I20260811 00:15:43.714458 31149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:43.670562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41469 (local address 127.7.94.3:34159)
0811 00:15:43.692323 (+ 21761us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:43.692390 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:43.694756 (+ 2366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:43.695592 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:43.695601 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:43.696161 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:43.696933 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:43.696959 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.698628 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.698633 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:43.700688 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:43.700703 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.701859 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.701868 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:43.702116 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:43.703281 (+ 1165us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:43.703315 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:43.705514 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:43.710563 (+ 5049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:43.710569 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:43.710581 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:43.710987 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:43.711390 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:43.711393 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:43.711395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:43.711594 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:43.712248 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:43.712254 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:43.712574 (+ 320us) client_negotiation.cc:769] Sending connection context
0811 00:15:43.712805 (+ 231us) client_negotiation.cc:240] Negotiation successful
0811 00:15:43.713180 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21441,"thread_start_us":21238,"threads_started":1}
I20260811 00:15:43.715901 31139 heartbeater.cc:347] Connected to a master server at 127.7.94.62:41469
I20260811 00:15:43.716214 31139 heartbeater.cc:464] Registering TS with master...
I20260811 00:15:43.716930 31139 heartbeater.cc:511] Master 127.7.94.62:41469 requested a full tablet report, sending...
I20260811 00:15:43.718416 30326 ts_manager.cc:194] Registered new tserver with Master: 5f312aea29ea41cd94e0c4c60e245975 (127.7.94.3:43811)
I20260811 00:15:43.719177 30326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:34159
I20260811 00:15:43.723034 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:15:43.741523 31154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:43.727322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37410 (local address 127.7.94.62:41469)
0811 00:15:43.727569 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:43.727577 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:43.727941 (+ 364us) server_negotiation.cc:407] Connection header received
0811 00:15:43.728131 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:43.728134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:43.728206 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:43.728288 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:43.730436 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.731126 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.733082 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.733566 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.733657 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:43.736255 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:43.736260 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:43.736263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:43.736314 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:43.739075 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:43.739705 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:43.739709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:43.739711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:43.739845 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:43.740205 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:43.740208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:43.740210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:43.740458 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:43.740654 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:43.740928 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 00:15:43.741112 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:15:43.746279 30326 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37410:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:15:43.753825 30326 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:15:43.779121 31172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:43.768000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51024 (local address 127.7.94.3:43811)
0811 00:15:43.769565 (+ 1565us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:43.769577 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:43.769610 (+ 33us) server_negotiation.cc:407] Connection header received
0811 00:15:43.769717 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:43.769726 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:43.770013 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:43.770205 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:15:43.771929 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.773661 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.776671 (+ 3010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.777890 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.778032 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:43.778270 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:43.778553 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 00:15:43.778724 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1383,"thread_start_us":1197,"threads_started":1}
I20260811 00:15:43.780197 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:43.770735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35152 (local address 127.7.94.1:44221)
0811 00:15:43.771960 (+ 1225us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:43.771969 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:43.771995 (+ 26us) server_negotiation.cc:407] Connection header received
0811 00:15:43.772096 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:43.772104 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:43.772327 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:43.772524 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:15:43.773161 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.775170 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.777462 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.778674 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.778799 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:43.779084 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:43.779684 (+ 600us) server_negotiation.cc:299] Negotiation successful
0811 00:15:43.779859 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1066,"thread_start_us":890,"threads_started":1}
I20260811 00:15:43.780194 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:43.769759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60742 (local address 127.7.94.2:37945)
0811 00:15:43.771583 (+ 1824us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:43.771591 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:43.771619 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:15:43.771737 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:43.771746 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:43.771980 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:43.772165 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:15:43.772862 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.774316 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.777332 (+ 3016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.778715 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.778841 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:43.779684 (+ 843us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:43.779763 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 00:15:43.779868 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1661,"thread_start_us":1493,"threads_started":1}
I20260811 00:15:43.780189 31171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:43.767924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:43811 (local address 127.0.0.1:51024)
0811 00:15:43.768755 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:43.768786 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:43.768937 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:43.770206 (+ 1269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:43.770212 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:43.770250 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:15:43.770740 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:43.770756 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.773835 (+ 3079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.773841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:43.776496 (+ 2655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:43.776512 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.778114 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.778120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:43.778308 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:43.778357 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 00:15:43.778512 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 00:15:43.779749 (+ 1237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":51,"thread_start_us":458,"threads_started":1}
I20260811 00:15:43.780655 31173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:43.769387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:37945 (local address 127.0.0.1:60742)
0811 00:15:43.769898 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:43.769916 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:43.770023 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:43.772158 (+ 2135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:43.772162 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:43.772186 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:15:43.772667 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:43.772681 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.774463 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.774468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:43.776508 (+ 2040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:43.776522 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.779114 (+ 2592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.779121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:43.779313 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:43.779341 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 00:15:43.779442 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 00:15:43.780246 (+ 804us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":683,"thread_start_us":268,"threads_started":1}
I20260811 00:15:43.781162 31179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:43.770031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:44221 (local address 127.0.0.1:35152)
0811 00:15:43.770927 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:43.770943 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:43.771055 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:43.772533 (+ 1478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:43.772539 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:43.772575 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:15:43.772970 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:43.772985 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.775336 (+ 2351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.775341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:43.777315 (+ 1974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:43.777331 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.779247 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.779253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:43.779448 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:43.779479 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 00:15:43.779567 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 00:15:43.780745 (+ 1178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"mutex_wait_us":1046,"thread_start_us":914,"threads_started":1}
I20260811 00:15:43.784098 31038 tablet_service.cc:1467] Processing CreateTablet for tablet c5968abd08bf43fcb3cdd2d3dfa55350 (DEFAULT_TABLE table=test-table [id=855288b829584b3993c7066808de1c06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:15:43.784597 30797 tablet_service.cc:1467] Processing CreateTablet for tablet c5968abd08bf43fcb3cdd2d3dfa55350 (DEFAULT_TABLE table=test-table [id=855288b829584b3993c7066808de1c06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:15:43.785615 31038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5968abd08bf43fcb3cdd2d3dfa55350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:15:43.785207 30553 tablet_service.cc:1467] Processing CreateTablet for tablet c5968abd08bf43fcb3cdd2d3dfa55350 (DEFAULT_TABLE table=test-table [id=855288b829584b3993c7066808de1c06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:15:43.786077 30797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5968abd08bf43fcb3cdd2d3dfa55350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:15:43.786638 30553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5968abd08bf43fcb3cdd2d3dfa55350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:15:43.799232 31182 tablet_bootstrap.cc:492] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975: Bootstrap starting.
I20260811 00:15:43.801743 31186 tablet_bootstrap.cc:492] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2: Bootstrap starting.
I20260811 00:15:43.802518 31183 tablet_bootstrap.cc:492] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd: Bootstrap starting.
I20260811 00:15:43.802536 31182 tablet_bootstrap.cc:654] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975: Neither blocks nor log segments found. Creating new log.
I20260811 00:15:43.803856 31182 log.cc:824] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975: Log is configured to *not* fsync() on all Append() calls
I20260811 00:15:43.806237 31183 tablet_bootstrap.cc:654] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd: Neither blocks nor log segments found. Creating new log.
I20260811 00:15:43.806859 31182 tablet_bootstrap.cc:492] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975: No bootstrap required, opened a new log
I20260811 00:15:43.807153 31182 ts_tablet_manager.cc:1397] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 00:15:43.807600 31183 log.cc:824] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd: Log is configured to *not* fsync() on all Append() calls
I20260811 00:15:43.808084 31186 tablet_bootstrap.cc:654] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2: Neither blocks nor log segments found. Creating new log.
I20260811 00:15:43.809204 31186 log.cc:824] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2: Log is configured to *not* fsync() on all Append() calls
I20260811 00:15:43.810797 31183 tablet_bootstrap.cc:492] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd: No bootstrap required, opened a new log
I20260811 00:15:43.811066 31183 ts_tablet_manager.cc:1397] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 00:15:43.812333 31186 tablet_bootstrap.cc:492] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2: No bootstrap required, opened a new log
I20260811 00:15:43.812595 31186 ts_tablet_manager.cc:1397] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.002s
I20260811 00:15:43.815317 31182 raft_consensus.cc:348] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f312aea29ea41cd94e0c4c60e245975" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43811 } } peers { permanent_uuid: "721ccc5670564e3ca67a18562fa48bfd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37945 } } peers { permanent_uuid: "bd0ad5916f9a417bbf48414e86678ab2" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 44221 } }
I20260811 00:15:43.815688 31182 raft_consensus.cc:374] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:15:43.815809 31182 raft_consensus.cc:729] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f312aea29ea41cd94e0c4c60e245975, State: Initialized, Role: FOLLOWER
I20260811 00:15:43.816444 31182 consensus_queue.cc:260] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f312aea29ea41cd94e0c4c60e245975" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43811 } } peers { permanent_uuid: "721ccc5670564e3ca67a18562fa48bfd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37945 } } peers { permanent_uuid: "bd0ad5916f9a417bbf48414e86678ab2" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 44221 } }
I20260811 00:15:43.818270 31182 ts_tablet_manager.cc:1428] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260811 00:15:43.818796 31139 heartbeater.cc:503] Master 127.7.94.62:41469 was elected leader, sending a full tablet report...
I20260811 00:15:43.823324 31186 raft_consensus.cc:348] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f312aea29ea41cd94e0c4c60e245975" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43811 } } peers { permanent_uuid: "721ccc5670564e3ca67a18562fa48bfd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37945 } } peers { permanent_uuid: "bd0ad5916f9a417bbf48414e86678ab2" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 44221 } }
I20260811 00:15:43.823323 31183 raft_consensus.cc:348] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f312aea29ea41cd94e0c4c60e245975" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43811 } } peers { permanent_uuid: "721ccc5670564e3ca67a18562fa48bfd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37945 } } peers { permanent_uuid: "bd0ad5916f9a417bbf48414e86678ab2" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 44221 } }
I20260811 00:15:43.823793 31183 raft_consensus.cc:374] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:15:43.823848 31186 raft_consensus.cc:374] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:15:43.823905 31183 raft_consensus.cc:729] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 721ccc5670564e3ca67a18562fa48bfd, State: Initialized, Role: FOLLOWER
I20260811 00:15:43.823980 31186 raft_consensus.cc:729] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd0ad5916f9a417bbf48414e86678ab2, State: Initialized, Role: FOLLOWER
I20260811 00:15:43.824568 31183 consensus_queue.cc:260] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f312aea29ea41cd94e0c4c60e245975" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43811 } } peers { permanent_uuid: "721ccc5670564e3ca67a18562fa48bfd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37945 } } peers { permanent_uuid: "bd0ad5916f9a417bbf48414e86678ab2" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 44221 } }
I20260811 00:15:43.824565 31186 consensus_queue.cc:260] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f312aea29ea41cd94e0c4c60e245975" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43811 } } peers { permanent_uuid: "721ccc5670564e3ca67a18562fa48bfd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37945 } } peers { permanent_uuid: "bd0ad5916f9a417bbf48414e86678ab2" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 44221 } }
I20260811 00:15:43.826514 31183 ts_tablet_manager.cc:1428] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260811 00:15:43.826501 31186 ts_tablet_manager.cc:1428] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260811 00:15:43.826949 30664 heartbeater.cc:503] Master 127.7.94.62:41469 was elected leader, sending a full tablet report...
W20260811 00:15:43.833950 30904 tablet.cc:2369] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:15:43.899413 30665 tablet.cc:2369] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:15:43.922206 31146 tablet.cc:2369] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:15:43.922328 31195 raft_consensus.cc:482] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:15:43.922669 31195 raft_consensus.cc:504] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f312aea29ea41cd94e0c4c60e245975" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43811 } } peers { permanent_uuid: "721ccc5670564e3ca67a18562fa48bfd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37945 } } peers { permanent_uuid: "bd0ad5916f9a417bbf48414e86678ab2" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 44221 } }
I20260811 00:15:43.924455 31195 leader_election.cc:288] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f312aea29ea41cd94e0c4c60e245975 (127.7.94.3:43811), 721ccc5670564e3ca67a18562fa48bfd (127.7.94.2:37945)
I20260811 00:15:43.930980 31172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:43.924997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:56113 (local address 127.7.94.3:43811)
0811 00:15:43.925196 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:43.925201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:43.925661 (+ 460us) server_negotiation.cc:407] Connection header received
0811 00:15:43.925881 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:43.925888 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:43.925999 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:43.926117 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:15:43.926810 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.927544 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.929290 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.930021 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.930208 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:43.930315 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:43.930597 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 00:15:43.930722 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 00:15:43.931143 31200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:43.924801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:43811 (local address 127.7.94.1:56113)
0811 00:15:43.925555 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:43.925575 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:43.925714 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:43.926138 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:43.926143 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:43.926173 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:15:43.926660 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:43.926670 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.927686 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.927692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:43.929130 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:43.929143 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.930264 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.930270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:43.930432 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:43.930455 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 00:15:43.930557 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 00:15:43.930697 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"mutex_wait_us":18,"spinlock_wait_cycles":18176,"thread_start_us":382,"threads_started":1}
I20260811 00:15:43.931440 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:43.925812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:43397 (local address 127.7.94.2:37945)
0811 00:15:43.926029 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:43.926036 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:43.926372 (+ 336us) server_negotiation.cc:407] Connection header received
0811 00:15:43.926576 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:43.926581 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:43.926648 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:43.926749 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:15:43.927293 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.927992 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.929739 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.930510 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.930673 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:43.930779 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:43.931073 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 00:15:43.931198 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 00:15:43.931573 31202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:43.925637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:37945 (local address 127.7.94.1:43397)
0811 00:15:43.926273 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:43.926290 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:43.926436 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:43.926765 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:43.926769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:43.926792 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:15:43.927139 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:43.927148 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.928131 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.928142 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:43.929587 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:43.929600 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.930732 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.930738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:43.930913 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:43.930935 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 00:15:43.931043 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 00:15:43.931182 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":19,"thread_start_us":385,"threads_started":1}
I20260811 00:15:43.932399 31066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5968abd08bf43fcb3cdd2d3dfa55350" candidate_uuid: "bd0ad5916f9a417bbf48414e86678ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f312aea29ea41cd94e0c4c60e245975" is_pre_election: true
I20260811 00:15:43.932668 30823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5968abd08bf43fcb3cdd2d3dfa55350" candidate_uuid: "bd0ad5916f9a417bbf48414e86678ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "721ccc5670564e3ca67a18562fa48bfd" is_pre_election: true
I20260811 00:15:43.933048 31066 raft_consensus.cc:2454] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd0ad5916f9a417bbf48414e86678ab2 in term 0.
I20260811 00:15:43.933229 30823 raft_consensus.cc:2454] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd0ad5916f9a417bbf48414e86678ab2 in term 0.
I20260811 00:15:43.933760 30471 leader_election.cc:302] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f312aea29ea41cd94e0c4c60e245975, bd0ad5916f9a417bbf48414e86678ab2; no voters:
I20260811 00:15:43.934163 31195 raft_consensus.cc:2781] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:15:43.934324 31195 raft_consensus.cc:482] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:15:43.934433 31195 raft_consensus.cc:3037] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:15:43.936642 31195 raft_consensus.cc:504] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f312aea29ea41cd94e0c4c60e245975" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43811 } } peers { permanent_uuid: "721ccc5670564e3ca67a18562fa48bfd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37945 } } peers { permanent_uuid: "bd0ad5916f9a417bbf48414e86678ab2" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 44221 } }
I20260811 00:15:43.937198 31195 leader_election.cc:288] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [CANDIDATE]: Term 1 election: Requested vote from peers 5f312aea29ea41cd94e0c4c60e245975 (127.7.94.3:43811), 721ccc5670564e3ca67a18562fa48bfd (127.7.94.2:37945)
I20260811 00:15:43.937666 31066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5968abd08bf43fcb3cdd2d3dfa55350" candidate_uuid: "bd0ad5916f9a417bbf48414e86678ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f312aea29ea41cd94e0c4c60e245975"
I20260811 00:15:43.937762 30823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5968abd08bf43fcb3cdd2d3dfa55350" candidate_uuid: "bd0ad5916f9a417bbf48414e86678ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "721ccc5670564e3ca67a18562fa48bfd"
I20260811 00:15:43.937881 31066 raft_consensus.cc:3037] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:15:43.937976 30823 raft_consensus.cc:3037] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:15:43.940083 31066 raft_consensus.cc:2454] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd0ad5916f9a417bbf48414e86678ab2 in term 1.
I20260811 00:15:43.940201 30823 raft_consensus.cc:2454] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd0ad5916f9a417bbf48414e86678ab2 in term 1.
I20260811 00:15:43.940541 30471 leader_election.cc:302] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f312aea29ea41cd94e0c4c60e245975, bd0ad5916f9a417bbf48414e86678ab2; no voters:
I20260811 00:15:43.940830 31195 raft_consensus.cc:2781] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:15:43.941370 31195 raft_consensus.cc:686] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [term 1 LEADER]: Becoming Leader. State: Replica: bd0ad5916f9a417bbf48414e86678ab2, State: Running, Role: LEADER
I20260811 00:15:43.941862 31195 consensus_queue.cc:237] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f312aea29ea41cd94e0c4c60e245975" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43811 } } peers { permanent_uuid: "721ccc5670564e3ca67a18562fa48bfd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37945 } } peers { permanent_uuid: "bd0ad5916f9a417bbf48414e86678ab2" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 44221 } }
I20260811 00:15:43.946877 30326 catalog_manager.cc:5495] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd0ad5916f9a417bbf48414e86678ab2 (127.7.94.1). New cstate: current_term: 1 leader_uuid: "bd0ad5916f9a417bbf48414e86678ab2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f312aea29ea41cd94e0c4c60e245975" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "721ccc5670564e3ca67a18562fa48bfd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd0ad5916f9a417bbf48414e86678ab2" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 44221 } health_report { overall_health: HEALTHY } } }
I20260811 00:15:43.970270 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:43.965419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35168 (local address 127.7.94.1:44221)
0811 00:15:43.965591 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:43.965595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:43.965611 (+ 16us) server_negotiation.cc:407] Connection header received
0811 00:15:43.965709 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:43.965713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:43.965785 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:43.965906 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:15:43.966214 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.966888 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.968165 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:43.968534 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:43.968622 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:43.969072 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:15:43.969679 (+ 607us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:15:43.969789 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:43.969900 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 00:15:43.969966 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 00:15:43.978780 30823 raft_consensus.cc:1261] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd [term 1 FOLLOWER]: Refusing update from remote peer bd0ad5916f9a417bbf48414e86678ab2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:15:43.978780 31066 raft_consensus.cc:1261] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975 [term 1 FOLLOWER]: Refusing update from remote peer bd0ad5916f9a417bbf48414e86678ab2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:15:43.979759 31195 consensus_queue.cc:1035] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f312aea29ea41cd94e0c4c60e245975" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:15:43.980080 31203 consensus_queue.cc:1035] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "721ccc5670564e3ca67a18562fa48bfd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:15:43.992228 31206 mvcc.cc:205] Tried to move back new op lower bound from 7317124480925974528 to 7317124480790261760. Current Snapshot: MvccSnapshot[applied={T|T < 7317124480925974528}]
I20260811 00:15:44.008612 30326 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:15:44.008920 30326 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:15:44.012637 30326 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb40369aef934468b4d1e72fdbabec0e: Sending DeleteTablet for 3 replicas of tablet c5968abd08bf43fcb3cdd2d3dfa55350
I20260811 00:15:44.013898 31038 tablet_service.cc:1514] Processing DeleteTablet for tablet c5968abd08bf43fcb3cdd2d3dfa55350 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:15:44 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:51024
I20260811 00:15:44.014024 30553 tablet_service.cc:1514] Processing DeleteTablet for tablet c5968abd08bf43fcb3cdd2d3dfa55350 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:15:44 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:35152
I20260811 00:15:44.014071 30797 tablet_service.cc:1514] Processing DeleteTablet for tablet c5968abd08bf43fcb3cdd2d3dfa55350 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:15:44 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:60742
I20260811 00:15:44.023630 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 30387
I20260811 00:15:44.026726 31226 tablet_replica.cc:330] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2: stopping tablet replica
I20260811 00:15:44.026947 31224 tablet_replica.cc:330] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975: stopping tablet replica
I20260811 00:15:44.027200 31226 raft_consensus.cc:2229] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:15:44.027503 31224 raft_consensus.cc:2229] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:15:44.027968 31224 raft_consensus.cc:2258] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:15:44.027922 31226 raft_consensus.cc:2258] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:15:44.030115 31224 ts_tablet_manager.cc:1905] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:15:44.031688 31228 tablet_replica.cc:330] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd: stopping tablet replica
I20260811 00:15:44.032132 31228 raft_consensus.cc:2229] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:15:44.032660 31228 raft_consensus.cc:2258] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:15:44.034926 31228 ts_tablet_manager.cc:1905] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:15:44.034930 31226 ts_tablet_manager.cc:1905] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:15:44.039515 31224 ts_tablet_manager.cc:1918] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:15:44.039722 31224 log.cc:1196] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/c5968abd08bf43fcb3cdd2d3dfa55350
I20260811 00:15:44.040254 31224 ts_tablet_manager.cc:1939] T c5968abd08bf43fcb3cdd2d3dfa55350 P 5f312aea29ea41cd94e0c4c60e245975: Deleting consensus metadata
I20260811 00:15:44.042147 30295 catalog_manager.cc:4841] TS 5f312aea29ea41cd94e0c4c60e245975 (127.7.94.3:43811): tablet c5968abd08bf43fcb3cdd2d3dfa55350 (table test-table [id=855288b829584b3993c7066808de1c06]) successfully deleted
I20260811 00:15:44.042439 31226 ts_tablet_manager.cc:1918] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:15:44.042622 31226 log.cc:1196] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/c5968abd08bf43fcb3cdd2d3dfa55350
I20260811 00:15:44.043285 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:44.024747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35176 (local address 127.7.94.1:44221)
0811 00:15:44.024977 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:44.024986 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:44.025015 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:15:44.025197 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:44.025205 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:44.025340 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:44.025483 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:44.027254 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:44.028492 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:44.030412 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:44.031269 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:44.031409 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:44.034932 (+ 3523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:44.034945 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:44.034963 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:44.035029 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:44.039277 (+ 4248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:44.040374 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:44.040383 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:44.040387 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:44.040567 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:44.041004 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:44.041009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:44.041012 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:44.041494 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:44.041785 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:44.042115 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 00:15:44.042373 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 00:15:44.043309 31226 ts_tablet_manager.cc:1939] T c5968abd08bf43fcb3cdd2d3dfa55350 P bd0ad5916f9a417bbf48414e86678ab2: Deleting consensus metadata
I20260811 00:15:44.044064 30657 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:15:44.048290 31228 ts_tablet_manager.cc:1918] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:15:44.048456 31228 log.cc:1196] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/c5968abd08bf43fcb3cdd2d3dfa55350
I20260811 00:15:44.048997 31228 ts_tablet_manager.cc:1939] T c5968abd08bf43fcb3cdd2d3dfa55350 P 721ccc5670564e3ca67a18562fa48bfd: Deleting consensus metadata
I20260811 00:15:44.050482 30295 catalog_manager.cc:4841] TS 721ccc5670564e3ca67a18562fa48bfd (127.7.94.2:37945): tablet c5968abd08bf43fcb3cdd2d3dfa55350 (table test-table [id=855288b829584b3993c7066808de1c06]) successfully deleted
I20260811 00:15:44.138531 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 30387
I20260811 00:15:44.138648 30295 catalog_manager.cc:4841] TS bd0ad5916f9a417bbf48414e86678ab2 (127.7.94.1:44221): tablet c5968abd08bf43fcb3cdd2d3dfa55350 (table test-table [id=855288b829584b3993c7066808de1c06]) successfully deleted
I20260811 00:15:44.163492 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 30669
I20260811 00:15:44.177760 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:44.164299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60750 (local address 127.7.94.2:37945)
0811 00:15:44.164479 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:44.164485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:44.164506 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:15:44.164660 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:44.164666 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:44.164744 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:44.164843 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:44.166211 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:44.167353 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:44.168669 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:44.169550 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:44.169668 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:44.172277 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:44.172293 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:44.172310 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:44.172348 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:44.175186 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:44.175794 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:44.175801 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:44.175803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:44.175931 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:44.176238 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:44.176241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:44.176243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:44.176573 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:44.176801 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:44.177049 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 00:15:44.177299 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:15:44.178398 30893 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:15:44.268792 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 30669
I20260811 00:15:44.298712 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 30912
I20260811 00:15:44.311932 31172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:44.299538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51030 (local address 127.7.94.3:43811)
0811 00:15:44.299713 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:44.299717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:44.299736 (+ 19us) server_negotiation.cc:407] Connection header received
0811 00:15:44.299900 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:44.299904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:44.299968 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:44.300053 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:44.301408 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:44.302135 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:44.303396 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:44.303795 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:44.303884 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:44.306590 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:44.306602 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:44.306617 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:44.306664 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:44.309541 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:44.310189 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:44.310196 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:44.310198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:44.310324 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:44.310654 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:44.310657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:44.310660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:44.311012 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:44.311256 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:44.311478 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 00:15:44.311634 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 00:15:44.312621 31127 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:15:44.401592 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 30912
I20260811 00:15:44.427801 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 30265
I20260811 00:15:44.429073 30376 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:15:44.543047 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 30265
I20260811 00:15:44.577566 7544 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 00:15:44.924027 7544 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 00:15:44.924270 7544 mini_ranger.cc:66] Stopping Ranger...
I20260811 00:15:45.280790 7544 mini_ranger.cc:68] Stopped Ranger
2026-08-11 00:15:45.281 UTC [28544] LOG: received smart shutdown request
2026-08-11 00:15:45.284 UTC [28544] LOG: background worker "logical replication launcher" (PID 28557) exited with exit code 1
2026-08-11 00:15:45.284 UTC [28549] LOG: shutting down
2026-08-11 00:15:45.453 UTC [28544] LOG: database system is shut down
2026-08-11T00:15:45Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53654 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:15:45 dist-test-slave-8hhm krb5kdc[31243](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:15:45 dist-test-slave-8hhm krb5kdc[31243](info): set up 2 sockets
Aug 11 00:15:45 dist-test-slave-8hhm krb5kdc[31243](info): commencing operation
krb5kdc: starting...
W20260811 00:15:46.944911 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.310s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:15:47 dist-test-slave-8hhm krb5kdc[31243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:15:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:15:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:15:47.226389 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:45191
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:45191,127.7.94.61:41447,127.7.94.60:42723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:15:47.567644 31264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:15:47.567936 31264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:15:47.568024 31264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:15:47.574978 31264 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:15:47.575080 31264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:15:47.575133 31264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:15:47.575182 31264 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:15:47.575218 31264 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:15:47.584794 31264 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:45191,127.7.94.61:41447,127.7.94.60:42723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:45191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:15:47.586167 31264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:15:47.587484 31264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:15:47.599246 31271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:47.599164 31278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:47.599198 31275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:47.600085 31264 server_base.cc:1034] running on GCE node
I20260811 00:15:47.600770 31264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:15:47.602156 31264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:15:47.603359 31264 hybrid_clock.cc:648] HybridClock initialized: now 1786407347603321 us; error 23 us; skew 500 ppm
I20260811 00:15:47.603673 31264 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:15:47 dist-test-slave-8hhm krb5kdc[31243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407347, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:47.606696 31264 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:15:47.608016 31264 webserver.cc:460] Webserver started at http://127.7.94.62:42991/ using document root <none> and password file <none>
I20260811 00:15:47.608668 31264 fs_manager.cc:359] Metadata directory not provided
I20260811 00:15:47.608781 31264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:15:47.609009 31264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:15:47.611774 31264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "ebc2ed543b024be9983a3ef20c1442d5"
format_stamp: "Formatted at 2026-08-11 00:15:47 on dist-test-slave-8hhm"
server_key: "e2f60cde54d0b70075eda055e9f4d305"
server_key_iv: "e858d6a7b5024a74bd20becc6d35f6b5"
server_key_version: "encryptionkey@0"
I20260811 00:15:47.612534 31264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "ebc2ed543b024be9983a3ef20c1442d5"
format_stamp: "Formatted at 2026-08-11 00:15:47 on dist-test-slave-8hhm"
server_key: "e2f60cde54d0b70075eda055e9f4d305"
server_key_iv: "e858d6a7b5024a74bd20becc6d35f6b5"
server_key_version: "encryptionkey@0"
I20260811 00:15:47.617452 31264 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 00:15:47.620608 31288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:47.622066 31264 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:15:47.622200 31264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "ebc2ed543b024be9983a3ef20c1442d5"
format_stamp: "Formatted at 2026-08-11 00:15:47 on dist-test-slave-8hhm"
server_key: "e2f60cde54d0b70075eda055e9f4d305"
server_key_iv: "e858d6a7b5024a74bd20becc6d35f6b5"
server_key_version: "encryptionkey@0"
I20260811 00:15:47.622355 31264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:15:47.649971 31264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:15:47.653918 31264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:15:47.654227 31264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:15:47.673009 31264 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:45191
I20260811 00:15:47.673040 31387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:45191 every 8 connection(s)
I20260811 00:15:47.674420 31264 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:15:47.678143 31390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:15:47.684787 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 31264
I20260811 00:15:47.685025 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:15:47.684549 31390 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:15:47.685773 7544 external_mini_cluster.cc:1428] Setting key c8dc26f47efa9d2a5fc78a7fc3def92f
Aug 11 00:15:47 dist-test-slave-8hhm krb5kdc[31243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407347, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:15:47.706321 31398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:47.687911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41210 (local address 127.7.94.62:45191)
0811 00:15:47.689668 (+ 1757us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:47.689676 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:47.689729 (+ 53us) server_negotiation.cc:407] Connection header received
0811 00:15:47.691248 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:47.691253 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:47.691458 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:47.691651 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:47.692936 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:47.694190 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:47.695505 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:47.696048 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:47.696249 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:47.699622 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:47.699629 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:47.699639 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:47.699666 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:47.702324 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:47.702990 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:47.703004 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:47.703007 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:47.703138 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:47.703529 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:47.703536 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:47.703539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:47.703992 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:47.704197 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:47.704679 (+ 482us) server_negotiation.cc:299] Negotiation successful
0811 00:15:47.704982 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1543,"thread_start_us":1343,"threads_started":1}
I20260811 00:15:47.706419 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:47.687910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:45191 (local address 127.0.0.1:41210)
0811 00:15:47.688836 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:47.688882 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:47.690876 (+ 1994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:47.691677 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:47.691684 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:47.692407 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:47.692738 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:47.692755 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:47.694355 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:47.694360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:47.695369 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:47.695380 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:47.696275 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:47.696282 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:47.696451 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:47.697132 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:47.697153 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:47.699417 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:47.702447 (+ 3030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:47.702459 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:47.702469 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:47.702865 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:47.703255 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:47.703262 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:47.703267 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:47.703427 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:47.704134 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:47.704141 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:47.704334 (+ 193us) client_negotiation.cc:769] Sending connection context
0811 00:15:47.704566 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 00:15:47.705098 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":129,"thread_start_us":447,"threads_started":1}
I20260811 00:15:47.709611 31390 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:15:47.710652 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:47.710326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:41447 (local address 127.0.0.1:46116)
0811 00:15:47.710466 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:47.710555 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:41447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 00:15:47.710934 31289 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.94.61:41447: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.7.94.61@KRBTEST.COM; defaulting to no policy
I20260811 00:15:47.712536 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:47.712196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:41447 (local address 127.0.0.1:46120)
0811 00:15:47.712352 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:47.712418 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:41447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 00:15:47.712747 31390 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.61:41447: Network error: Client connection negotiation failed: client connection to 127.7.94.61:41447: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.61.keytab.
Entry for principal kudu/127.7.94.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.61.keytab.
I20260811 00:15:47.744724 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.61:41447
--webserver_interface=127.7.94.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.61.keytab
--principal=kudu/127.7.94.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:45191,127.7.94.61:41447,127.7.94.60:42723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:15:47.762167 31390 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } attempt: 1
I20260811 00:15:47.763302 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:47.762867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:41447 (local address 127.0.0.1:46126)
0811 00:15:47.763061 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:47.763159 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:41447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 00:15:47.764425 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:47.764089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:41447 (local address 127.0.0.1:46140)
0811 00:15:47.764245 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:47.764309 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:41447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 00:15:47.764638 31390 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.61:41447: Network error: Client connection negotiation failed: client connection to 127.7.94.61:41447: connect: Connection refused (error 111)
I20260811 00:15:47.832907 31390 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } attempt: 2
I20260811 00:15:47.834209 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:47.833728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:41447 (local address 127.0.0.1:46148)
0811 00:15:47.833973 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:47.834063 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:41447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 00:15:47.835291 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:47.834935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:41447 (local address 127.0.0.1:46164)
0811 00:15:47.835076 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:47.835166 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:41447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 00:15:47.835486 31390 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.61:41447: Network error: Client connection negotiation failed: client connection to 127.7.94.61:41447: connect: Connection refused (error 111)
I20260811 00:15:47.926743 31390 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } attempt: 3
I20260811 00:15:47.928018 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:47.927579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:41447 (local address 127.0.0.1:46174)
0811 00:15:47.927771 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:47.927871 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:41447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 00:15:47.929056 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:47.928756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:41447 (local address 127.0.0.1:46188)
0811 00:15:47.928874 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:47.928935 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:41447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 00:15:47.929325 31390 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.61:41447: Network error: Client connection negotiation failed: client connection to 127.7.94.61:41447: connect: Connection refused (error 111)
I20260811 00:15:48.072582 31390 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } attempt: 4
I20260811 00:15:48.073913 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.073370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:41447 (local address 127.0.0.1:46190)
0811 00:15:48.073691 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.073778 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:41447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260811 00:15:48.074934 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.074595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:41447 (local address 127.0.0.1:46206)
0811 00:15:48.074718 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.074821 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:41447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 00:15:48.075230 31390 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.61:41447: Network error: Client connection negotiation failed: client connection to 127.7.94.61:41447: connect: Connection refused (error 111)
W20260811 00:15:48.082952 31402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:15:48.083294 31402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:15:48.083451 31402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:15:48.090703 31402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:15:48.090854 31402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:15:48.090955 31402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:15:48.091050 31402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:15:48.091135 31402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:15:48.101085 31402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:45191,127.7.94.61:41447,127.7.94.60:42723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.61.keytab
--principal=kudu/127.7.94.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.61:41447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.94.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:15:48.102485 31402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:15:48.103899 31402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:15:48.115952 31410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:48.116075 31411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:48.115989 31413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:48.116986 31402 server_base.cc:1034] running on GCE node
I20260811 00:15:48.117762 31402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:15:48.119035 31402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:15:48.120224 31402 hybrid_clock.cc:648] HybridClock initialized: now 1786407348120182 us; error 33 us; skew 500 ppm
I20260811 00:15:48.120571 31402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:15:48 dist-test-slave-8hhm krb5kdc[31243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407348, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:48.123728 31402 init.cc:377] Logged in from keytab as kudu/127.7.94.61@KRBTEST.COM (short username kudu)
I20260811 00:15:48.125169 31402 webserver.cc:460] Webserver started at http://127.7.94.61:43713/ using document root <none> and password file <none>
I20260811 00:15:48.125963 31402 fs_manager.cc:359] Metadata directory not provided
I20260811 00:15:48.126106 31402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:15:48.126401 31402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:15:48.129350 31402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-1/data/instance:
uuid: "6acf6f65cad74985910f3a355fa67bbd"
format_stamp: "Formatted at 2026-08-11 00:15:48 on dist-test-slave-8hhm"
server_key: "f7413ec8fa6d7f1ae3abf18eb51eb668"
server_key_iv: "1798167e2b8631a3d4e113b5c7cef8e0"
server_key_version: "encryptionkey@0"
I20260811 00:15:48.130134 31402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-1/wal/instance:
uuid: "6acf6f65cad74985910f3a355fa67bbd"
format_stamp: "Formatted at 2026-08-11 00:15:48 on dist-test-slave-8hhm"
server_key: "f7413ec8fa6d7f1ae3abf18eb51eb668"
server_key_iv: "1798167e2b8631a3d4e113b5c7cef8e0"
server_key_version: "encryptionkey@0"
I20260811 00:15:48.135186 31402 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260811 00:15:48.139670 31425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:48.140900 31402 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260811 00:15:48.141075 31402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-1/wal
uuid: "6acf6f65cad74985910f3a355fa67bbd"
format_stamp: "Formatted at 2026-08-11 00:15:48 on dist-test-slave-8hhm"
server_key: "f7413ec8fa6d7f1ae3abf18eb51eb668"
server_key_iv: "1798167e2b8631a3d4e113b5c7cef8e0"
server_key_version: "encryptionkey@0"
I20260811 00:15:48.141341 31402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:15:48.156342 31402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:15:48.160175 31402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:15:48.160391 31402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:15:48.179010 31402 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.61:41447
I20260811 00:15:48.179042 31508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.61:41447 every 8 connection(s)
I20260811 00:15:48.180394 31402 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-1/data/info.pb
I20260811 00:15:48.181697 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 31402
I20260811 00:15:48.181905 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-1/wal/instance
I20260811 00:15:48.182440 7544 external_mini_cluster.cc:1428] Setting key dd6b14e2d0475530c981dba49f349c42
I20260811 00:15:48.185150 31509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:15:48.195266 31509 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:15:48 dist-test-slave-8hhm krb5kdc[31243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407348, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.61@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.94.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.60@KRBTEST.COM" created.
I20260811 00:15:48.219076 31398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.198038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41220 (local address 127.7.94.62:45191)
0811 00:15:48.198278 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:48.198286 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:48.199488 (+ 1202us) server_negotiation.cc:407] Connection header received
0811 00:15:48.202471 (+ 2983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:48.202475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:48.202545 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:48.202633 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:48.204938 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.205678 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.208233 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.208677 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.208805 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.213032 (+ 4227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:48.213037 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:48.213040 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:48.213072 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:48.215871 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.216581 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.216589 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.216595 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.216741 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.217116 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.217124 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.217130 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.217560 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:48.217729 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:48.218417 (+ 688us) server_negotiation.cc:299] Negotiation successful
0811 00:15:48.218659 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 00:15:48.219738 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.198413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:45191 (local address 127.0.0.1:41220)
0811 00:15:48.199263 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.199312 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:48.202183 (+ 2871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:48.202938 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:48.202951 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:48.203761 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:48.204647 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:48.204686 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.205872 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.205880 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:48.208068 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:48.208087 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.208944 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.208954 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:48.209319 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.210123 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:48.210158 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:48.212821 (+ 2663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:48.216027 (+ 3206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:48.216034 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:48.216046 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:48.216460 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:48.216851 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:48.216854 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:48.216856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:48.217005 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:48.217701 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:48.217706 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:48.218035 (+ 329us) client_negotiation.cc:769] Sending connection context
0811 00:15:48.218272 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 00:15:48.218636 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":343,"threads_started":1}
I20260811 00:15:48.221621 31509 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:15:48 dist-test-slave-8hhm krb5kdc[31243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407348, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.61@KRBTEST.COM for kudu/127.7.94.61@KRBTEST.COM
I20260811 00:15:48.237722 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.222466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:41447 (local address 127.0.0.1:46210)
0811 00:15:48.222640 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.222674 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:48.222773 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:48.224059 (+ 1286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:48.224062 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:48.224428 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:48.224567 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:48.224574 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.226686 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.226693 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:48.228129 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:48.228146 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.228956 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.228968 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:48.229216 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.229805 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:48.229831 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:48.232194 (+ 2363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:48.235310 (+ 3116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:48.235318 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:48.235322 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:48.235696 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:48.236049 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:48.236055 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:48.236058 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:48.236191 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:48.236747 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:48.236751 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:48.236900 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 00:15:48.237048 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 00:15:48.237320 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 00:15:48.237782 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.222759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46210 (local address 127.7.94.61:41447)
0811 00:15:48.223521 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:48.223531 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:48.223563 (+ 32us) server_negotiation.cc:407] Connection header received
0811 00:15:48.223621 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:48.223632 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:48.223877 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:48.224057 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:48.224721 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.226497 (+ 1776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.228285 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.228796 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.228903 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.232395 (+ 3492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:48.232406 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:48.232418 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:48.232445 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:48.235185 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.235824 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.235835 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.235838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.235940 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.236291 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.236297 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.236300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.236622 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:48.236839 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:48.237189 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 00:15:48.237412 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":406,"threads_started":1}
I20260811 00:15:48.240576 31509 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:15:48.241782 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.241179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40640)
0811 00:15:48.241395 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.241551 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260811 00:15:48.242066 31426 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
I20260811 00:15:48.243728 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.243379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40644)
0811 00:15:48.243543 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.243618 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 00:15:48.243978 31509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:42723: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.60.keytab.
Entry for principal kudu/127.7.94.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.60.keytab.
I20260811 00:15:48.250936 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.60:42723
--webserver_interface=127.7.94.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.60.keytab
--principal=kudu/127.7.94.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:45191,127.7.94.61:41447,127.7.94.60:42723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:15:48.293315 31509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } attempt: 1
I20260811 00:15:48.294524 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.294075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40652)
0811 00:15:48.294282 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.294379 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 00:15:48.295491 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.295154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40656)
0811 00:15:48.295294 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.295389 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 00:15:48.295686 31509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:42723: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
I20260811 00:15:48.363899 31509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } attempt: 2
I20260811 00:15:48.365106 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.364653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40662)
0811 00:15:48.364846 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.364945 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 00:15:48.366240 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.365904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40674)
0811 00:15:48.366049 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.366117 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 00:15:48.366447 31509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:42723: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
I20260811 00:15:48.374536 31390 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } attempt: 5
Aug 11 00:15:48 dist-test-slave-8hhm krb5kdc[31243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407347, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for kudu/127.7.94.61@KRBTEST.COM
I20260811 00:15:48.388357 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.375248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46220 (local address 127.7.94.61:41447)
0811 00:15:48.375439 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:48.375446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:48.375472 (+ 26us) server_negotiation.cc:407] Connection header received
0811 00:15:48.375544 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:48.375551 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:48.375660 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:48.375790 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:48.376589 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.377821 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.378918 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.379538 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.379664 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.383007 (+ 3343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:48.383012 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:48.383015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:48.383046 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:48.385632 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.386430 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.386434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.386436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.386564 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.387008 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.387011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.387013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.387284 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:48.387391 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:48.387936 (+ 545us) server_negotiation.cc:299] Negotiation successful
0811 00:15:48.388102 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 00:15:48.388564 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.375154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:41447 (local address 127.0.0.1:46220)
0811 00:15:48.375319 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.375339 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:48.375445 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:48.375811 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:48.375814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:48.376206 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:48.376391 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:48.376399 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.377964 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.377967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:48.378781 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:48.378792 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.379668 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.379672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:48.379818 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.380429 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:48.380446 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:48.382817 (+ 2371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:48.385792 (+ 2975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:48.385797 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:48.385801 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:48.386311 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:48.386675 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:48.386680 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:48.386684 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:48.386884 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:48.387416 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:48.387423 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:48.387645 (+ 222us) client_negotiation.cc:769] Sending connection context
0811 00:15:48.387835 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 00:15:48.388204 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 00:15:48.389667 31390 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:15:48.390710 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.390271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40688)
0811 00:15:48.390444 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.390538 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 00:15:48.391659 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.391292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40700)
0811 00:15:48.391445 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.391541 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 00:15:48.391904 31390 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:42723: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
I20260811 00:15:48.443171 31390 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } attempt: 1
I20260811 00:15:48.444525 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.443958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40702)
0811 00:15:48.444200 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.444309 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 00:15:48.445528 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.445246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40712)
0811 00:15:48.445366 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.445434 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 00:15:48.445811 31390 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:42723: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
I20260811 00:15:48.457651 31509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } attempt: 3
I20260811 00:15:48.458680 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.458231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40726)
0811 00:15:48.458447 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.458529 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 00:15:48.459681 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.459277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40742)
0811 00:15:48.459498 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.459574 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260811 00:15:48.459877 31509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:42723: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
I20260811 00:15:48.514067 31390 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } attempt: 2
I20260811 00:15:48.515256 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.514762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40746)
0811 00:15:48.514984 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.515069 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 00:15:48.516239 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.515917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40758)
0811 00:15:48.516063 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.516137 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 00:15:48.516497 31390 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:42723: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
W20260811 00:15:48.594681 31522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:15:48.595036 31522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:15:48.595193 31522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:15:48.602228 31522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:15:48.602363 31522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:15:48.602469 31522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:15:48.602569 31522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:15:48.602653 31522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:15:48.603078 31509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } attempt: 4
I20260811 00:15:48.604199 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.603709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40764)
0811 00:15:48.603955 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.604046 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 00:15:48.605108 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.604810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40774)
0811 00:15:48.604942 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.605013 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 00:15:48.605325 31509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:42723: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
I20260811 00:15:48.612015 31522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:45191,127.7.94.61:41447,127.7.94.60:42723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.60.keytab
--principal=kudu/127.7.94.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.60:42723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.94.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:15:48.613387 31522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:15:48.614656 31522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 00:15:48.622722 31390 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } attempt: 3
I20260811 00:15:48.623860 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.623404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40776)
0811 00:15:48.623629 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.623713 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 00:15:48.624919 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.624533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40786)
0811 00:15:48.624704 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.624795 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 00:15:48.625244 31390 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:42723: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42723: connect: Connection refused (error 111)
W20260811 00:15:48.627550 31535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:48.627624 31530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:48.627637 31532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:48.634730 31522 server_base.cc:1034] running on GCE node
I20260811 00:15:48.635520 31522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:15:48.636894 31522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:15:48.638146 31522 hybrid_clock.cc:648] HybridClock initialized: now 1786407348638075 us; error 56 us; skew 500 ppm
I20260811 00:15:48.638502 31522 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:15:48 dist-test-slave-8hhm krb5kdc[31243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407348, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:48.641696 31522 init.cc:377] Logged in from keytab as kudu/127.7.94.60@KRBTEST.COM (short username kudu)
I20260811 00:15:48.643237 31522 webserver.cc:460] Webserver started at http://127.7.94.60:37817/ using document root <none> and password file <none>
I20260811 00:15:48.643962 31522 fs_manager.cc:359] Metadata directory not provided
I20260811 00:15:48.644093 31522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:15:48.644405 31522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:15:48.647472 31522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-2/data/instance:
uuid: "91b81de49d374083a692a5fcbfb878d0"
format_stamp: "Formatted at 2026-08-11 00:15:48 on dist-test-slave-8hhm"
server_key: "9357868e0af7e4129ab75e3a8fa6e30f"
server_key_iv: "6d97dbeee79f6fce0b96459395d21f4d"
server_key_version: "encryptionkey@0"
I20260811 00:15:48.648296 31522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-2/wal/instance:
uuid: "91b81de49d374083a692a5fcbfb878d0"
format_stamp: "Formatted at 2026-08-11 00:15:48 on dist-test-slave-8hhm"
server_key: "9357868e0af7e4129ab75e3a8fa6e30f"
server_key_iv: "6d97dbeee79f6fce0b96459395d21f4d"
server_key_version: "encryptionkey@0"
I20260811 00:15:48.653367 31522 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 00:15:48.656564 31545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:48.657989 31522 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:15:48.658135 31522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-2/wal
uuid: "91b81de49d374083a692a5fcbfb878d0"
format_stamp: "Formatted at 2026-08-11 00:15:48 on dist-test-slave-8hhm"
server_key: "9357868e0af7e4129ab75e3a8fa6e30f"
server_key_iv: "6d97dbeee79f6fce0b96459395d21f4d"
server_key_version: "encryptionkey@0"
I20260811 00:15:48.658311 31522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:15:48.678977 31522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:15:48.684024 31522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:15:48.684266 31522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:15:48.702582 31522 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.60:42723
I20260811 00:15:48.702616 31628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.60:42723 every 8 connection(s)
I20260811 00:15:48.703994 31522 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-2/data/info.pb
I20260811 00:15:48.707535 31629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:15:48.708483 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 31522
I20260811 00:15:48.708657 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786407133955383-7544-0/minicluster-data/master-2/wal/instance
I20260811 00:15:48.709188 7544 external_mini_cluster.cc:1428] Setting key b97daca420ddce38b09d7410a58cc925
I20260811 00:15:48.716539 31629 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:15:48 dist-test-slave-8hhm krb5kdc[31243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.60@KRBTEST.COM
Aug 11 00:15:48 dist-test-slave-8hhm krb5kdc[31243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407348, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.60@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:15:48.732635 31633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.710981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40794 (local address 127.7.94.60:42723)
0811 00:15:48.711779 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:48.711810 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:48.711874 (+ 64us) server_negotiation.cc:407] Connection header received
0811 00:15:48.713528 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:48.713558 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:48.713934 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:48.714332 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:48.716028 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.717326 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.719178 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.720191 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.720371 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.725119 (+ 4748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:48.725126 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:48.725136 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:48.725220 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:48.728562 (+ 3342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.729280 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.729288 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.729300 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.729473 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.729952 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.729956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.729958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.730509 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:48.730775 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:48.731262 (+ 487us) server_negotiation.cc:299] Negotiation successful
0811 00:15:48.731619 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":538,"threads_started":1}
I20260811 00:15:48.734779 31398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.719314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41236 (local address 127.7.94.62:45191)
0811 00:15:48.719522 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:48.719528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:48.720325 (+ 797us) server_negotiation.cc:407] Connection header received
0811 00:15:48.720495 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:48.720501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:48.720583 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:48.720688 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:48.721835 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.722986 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.724679 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.725126 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.725305 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.728889 (+ 3584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:48.728894 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:48.728897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:48.728927 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:48.731982 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.732681 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.732688 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.732691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.732892 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.733319 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.733326 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.733329 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.733574 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:48.733748 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:48.734219 (+ 471us) server_negotiation.cc:299] Negotiation successful
0811 00:15:48.734410 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:15:48.734854 31637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.719370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:45191 (local address 127.0.0.1:41236)
0811 00:15:48.720186 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.720235 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:48.720370 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:48.720746 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:48.720751 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:48.721455 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:48.721691 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:48.721699 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.723165 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.723169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:48.724528 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:48.724541 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.725383 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.725392 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:48.725586 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.726421 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:48.726438 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:48.728701 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:48.732121 (+ 3420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:48.732131 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:48.732138 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:48.732550 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:48.733019 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:48.733025 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:48.733029 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:48.733171 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:48.733715 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:48.733723 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:48.733918 (+ 195us) client_negotiation.cc:769] Sending connection context
0811 00:15:48.734113 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 00:15:48.734391 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":421,"threads_started":1}
I20260811 00:15:48.736322 31629 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:15:48 dist-test-slave-8hhm krb5kdc[31243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407348, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.60@KRBTEST.COM for kudu/127.7.94.61@KRBTEST.COM
Aug 11 00:15:48 dist-test-slave-8hhm krb5kdc[31243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:15:48.749641 31637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.737001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:41447 (local address 127.0.0.1:46232)
0811 00:15:48.737179 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.737196 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:48.737325 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:48.737683 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:48.737687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:48.738032 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:48.738240 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:48.738253 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.739264 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.739267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:48.740082 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:48.740092 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.740874 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.740880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:48.741036 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.741691 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:48.741706 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:48.743567 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:48.747211 (+ 3644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:48.747218 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:48.747222 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:48.747723 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:48.748128 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:48.748135 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:48.748141 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:48.748329 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:48.748865 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:48.748871 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:48.749035 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 00:15:48.749204 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 00:15:48.749428 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 00:15:48.749929 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.737197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46232 (local address 127.7.94.61:41447)
0811 00:15:48.737431 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:48.737444 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:48.737458 (+ 14us) server_negotiation.cc:407] Connection header received
0811 00:15:48.737505 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:48.737509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:48.737563 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:48.737665 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:48.738394 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.739108 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.740234 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.740682 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.740827 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.743759 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:48.743764 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:48.743767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:48.743797 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:48.746984 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.747845 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.747848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.747851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.747975 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.748434 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.748437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.748439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.748664 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:48.748805 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:48.749328 (+ 523us) server_negotiation.cc:299] Negotiation successful
0811 00:15:48.749485 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260811 00:15:48.750890 31629 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:15:48.752645 31398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.736260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41252 (local address 127.7.94.62:45191)
0811 00:15:48.736403 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:48.736410 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:48.736501 (+ 91us) server_negotiation.cc:407] Connection header received
0811 00:15:48.736654 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:48.736659 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:48.736729 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:48.736811 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:48.738114 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.738919 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.740270 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.740654 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.740740 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.745059 (+ 4319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:48.745063 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:48.745066 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:48.745104 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:48.748445 (+ 3341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.750159 (+ 1714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.750163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.750165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.750276 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.750721 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.750726 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.750729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.751029 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:48.751296 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:48.752271 (+ 975us) server_negotiation.cc:299] Negotiation successful
0811 00:15:48.752426 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 00:15:48 dist-test-slave-8hhm krb5kdc[31243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407348, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.60@KRBTEST.COM for kudu/127.7.94.60@KRBTEST.COM
Aug 11 00:15:48 dist-test-slave-8hhm krb5kdc[31243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.61@KRBTEST.COM
I20260811 00:15:48.765172 31637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.751537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40800)
0811 00:15:48.751758 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.751775 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:48.751876 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:48.752335 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:48.752339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:48.752672 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:48.752832 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:48.752842 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.754039 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.754042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:48.755044 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:48.755056 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.755789 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.755795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:48.755953 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.756508 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:48.756530 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:48.758895 (+ 2365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:48.762669 (+ 3774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:48.762673 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:48.762676 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:48.763080 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:48.763464 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:48.763469 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:48.763472 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:48.763646 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:48.764158 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:48.764162 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:48.764298 (+ 136us) client_negotiation.cc:769] Sending connection context
0811 00:15:48.764452 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 00:15:48.764749 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 00:15:48.765779 31633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.751781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40800 (local address 127.7.94.60:42723)
0811 00:15:48.751951 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:48.751958 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:48.751972 (+ 14us) server_negotiation.cc:407] Connection header received
0811 00:15:48.752007 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:48.752012 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:48.752069 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:48.752177 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:48.752981 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.753908 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.755194 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.755614 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.755741 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.759076 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:48.759081 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:48.759084 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:48.759112 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:48.762544 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.763215 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.763219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.763221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.763350 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.763757 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.763765 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.763768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.764020 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:48.764191 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:48.765346 (+ 1155us) server_negotiation.cc:299] Negotiation successful
0811 00:15:48.765570 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 00:15:48.771531 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.754464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46248 (local address 127.7.94.61:41447)
0811 00:15:48.754848 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:48.754852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:48.754865 (+ 13us) server_negotiation.cc:407] Connection header received
0811 00:15:48.754901 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:48.754904 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:48.754960 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:48.755097 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:48.756284 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.757455 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.758875 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.759363 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.759457 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.764523 (+ 5066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:48.764529 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:48.764534 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:48.764575 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:48.768755 (+ 4180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.769535 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.769541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.769545 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.769723 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.770085 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.770089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.770093 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.770428 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:48.770719 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:48.770964 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 00:15:48.771214 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289}
I20260811 00:15:48.772097 31629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0: Bootstrap starting.
I20260811 00:15:48.776059 31629 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0: Neither blocks nor log segments found. Creating new log.
I20260811 00:15:48.776404 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:15:48.777453 31629 log.cc:824] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0: Log is configured to *not* fsync() on all Append() calls
I20260811 00:15:48.780902 31629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0: No bootstrap required, opened a new log
I20260811 00:15:48.793076 31629 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc2ed543b024be9983a3ef20c1442d5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } } peers { permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } } peers { permanent_uuid: "91b81de49d374083a692a5fcbfb878d0" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } }
I20260811 00:15:48.793584 31629 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:15:48.793735 31629 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b81de49d374083a692a5fcbfb878d0, State: Initialized, Role: FOLLOWER
I20260811 00:15:48.794595 31629 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc2ed543b024be9983a3ef20c1442d5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } } peers { permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } } peers { permanent_uuid: "91b81de49d374083a692a5fcbfb878d0" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } }
I20260811 00:15:48.801647 31650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc2ed543b024be9983a3ef20c1442d5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } } peers { permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } } peers { permanent_uuid: "91b81de49d374083a692a5fcbfb878d0" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } } }
I20260811 00:15:48.802098 31650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:15:48.803086 31390 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } attempt: 4
I20260811 00:15:48.804028 31398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.782220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41266 (local address 127.7.94.62:45191)
0811 00:15:48.782657 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:48.782665 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:48.783428 (+ 763us) server_negotiation.cc:407] Connection header received
0811 00:15:48.783637 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:48.783643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:48.783739 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:48.783844 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:48.786859 (+ 3015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.788050 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.790264 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.790948 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.791045 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.795797 (+ 4752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:48.795806 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:48.795812 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:48.795858 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:48.800207 (+ 4349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.801321 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.801329 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.801335 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.801513 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.801996 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.802003 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.802009 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.802384 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:48.802608 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:48.803416 (+ 808us) server_negotiation.cc:299] Negotiation successful
0811 00:15:48.803675 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269}
I20260811 00:15:48.804165 31629 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:15:48.804912 31515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.782407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46250 (local address 127.7.94.61:41447)
0811 00:15:48.782849 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:48.782857 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:48.784292 (+ 1435us) server_negotiation.cc:407] Connection header received
0811 00:15:48.784543 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:48.784547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:48.784628 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:48.784706 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:48.787640 (+ 2934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.788738 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.790883 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.791576 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.791713 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.795644 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:48.795654 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:48.795661 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:48.795716 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:48.800091 (+ 4375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.801399 (+ 1308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.801404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.801407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.801571 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.802074 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.802081 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.802086 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.802475 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:48.802695 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:48.803370 (+ 675us) server_negotiation.cc:299] Negotiation successful
0811 00:15:48.804534 (+ 1164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279}
I20260811 00:15:48.805330 31633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.784718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40812 (local address 127.7.94.60:42723)
0811 00:15:48.784919 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:48.784925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:48.786255 (+ 1330us) server_negotiation.cc:407] Connection header received
0811 00:15:48.786414 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:48.786419 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:48.786489 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:48.786584 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:48.788665 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.789900 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.792090 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.792861 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.792963 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.797080 (+ 4117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:48.797086 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:48.797091 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:48.797127 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:48.800784 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.801722 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.801728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.801733 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.801917 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.802358 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.802363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.802366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.802769 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:48.803014 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:48.804602 (+ 1588us) server_negotiation.cc:299] Negotiation successful
0811 00:15:48.804866 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 00:15:48 dist-test-slave-8hhm krb5kdc[31243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407347, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for kudu/127.7.94.60@KRBTEST.COM
I20260811 00:15:48.818841 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.803633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40814)
0811 00:15:48.803869 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.803887 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:48.803983 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:48.805867 (+ 1884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:48.805872 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:48.806336 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:48.806542 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:48.806553 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.807862 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.807867 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:48.808887 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:48.808898 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.809836 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.809842 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:48.810006 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.810575 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:48.810599 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:48.812672 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:48.816339 (+ 3667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:48.816345 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:48.816349 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:48.816684 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:48.817064 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:48.817067 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:48.817069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:48.817200 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:48.817847 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:48.817851 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:48.818001 (+ 150us) client_negotiation.cc:769] Sending connection context
0811 00:15:48.818152 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 00:15:48.818467 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260811 00:15:48.818835 31633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.803685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40814 (local address 127.7.94.60:42723)
0811 00:15:48.805553 (+ 1868us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:48.805558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:48.805576 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:15:48.805621 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:48.805626 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:48.805702 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:48.806710 (+ 1008us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:48.806836 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.807711 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.809026 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.809660 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.810293 (+ 633us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.812874 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:48.812880 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:48.812884 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:48.812919 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:48.816210 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.816802 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.816809 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.816812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.816936 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.817386 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.817391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.817401 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.817691 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:48.817852 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:48.818251 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 00:15:48.818472 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1775,"thread_start_us":13980,"threads_started":1}
I20260811 00:15:48.824574 31390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5: Bootstrap starting.
I20260811 00:15:48.828495 31390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5: Neither blocks nor log segments found. Creating new log.
I20260811 00:15:48.829931 31390 log.cc:824] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5: Log is configured to *not* fsync() on all Append() calls
I20260811 00:15:48.833230 31390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5: No bootstrap required, opened a new log
W20260811 00:15:48.833928 31674 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:15:48.834044 31674 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:15:48.830951 31522 master_runner.cc:186] Error getting master registration for 127.7.94.62:45191: OK, instance_id { permanent_uuid: "ebc2ed543b024be9983a3ef20c1442d5" instance_seqno: 1786407347667494 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 00:15:48.840397 31522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 00:15:48.844163 31390 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc2ed543b024be9983a3ef20c1442d5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } } peers { permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } } peers { permanent_uuid: "91b81de49d374083a692a5fcbfb878d0" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } }
I20260811 00:15:48.844540 31390 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:15:48.844673 31390 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebc2ed543b024be9983a3ef20c1442d5, State: Initialized, Role: FOLLOWER
I20260811 00:15:48.845360 31390 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc2ed543b024be9983a3ef20c1442d5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } } peers { permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } } peers { permanent_uuid: "91b81de49d374083a692a5fcbfb878d0" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } }
I20260811 00:15:48.846431 31682 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc2ed543b024be9983a3ef20c1442d5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } } peers { permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } } peers { permanent_uuid: "91b81de49d374083a692a5fcbfb878d0" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } } }
I20260811 00:15:48.847754 31390 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:15:48.848215 31682 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 00:15:48.860308 31702 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:15:48.860436 31702 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:15:48.857292 31264 master_runner.cc:186] Error getting master registration for 127.7.94.61:41447: OK, instance_id { permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" instance_seqno: 1786407348173692 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 00:15:48.869771 31264 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 00:15:48.904544 31509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } attempt: 5
W20260811 00:15:48.907183 31301 tablet.cc:2369] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 00:15:48 dist-test-slave-8hhm krb5kdc[31243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407348, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.61@KRBTEST.COM for kudu/127.7.94.60@KRBTEST.COM
I20260811 00:15:48.921787 31513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.905174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42723 (local address 127.0.0.1:40824)
0811 00:15:48.905462 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:48.905485 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:48.905627 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:48.906090 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:48.906097 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:48.906632 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:48.906855 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:48.906868 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.908353 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.908359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:48.909867 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:48.909896 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.910884 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.910891 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:48.911085 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.911829 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:48.911861 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:48.914745 (+ 2884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:48.918834 (+ 4089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:48.918839 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:48.918843 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:48.919351 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:48.919837 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:48.919842 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:48.919846 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:48.920010 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:48.920746 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:48.920751 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:48.920915 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 00:15:48.921059 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 00:15:48.921365 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260811 00:15:48.922217 31633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:48.905551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40824 (local address 127.7.94.60:42723)
0811 00:15:48.905744 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:48.905751 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:48.905788 (+ 37us) server_negotiation.cc:407] Connection header received
0811 00:15:48.905871 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:48.905874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:48.905944 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:48.906052 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:48.907026 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.908188 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.910028 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:48.910726 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:48.910853 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:48.914936 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:48.914943 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:48.914948 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:48.914987 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:48.918680 (+ 3693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.919500 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.919506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.919509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.919696 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:48.920146 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:48.920152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:48.920155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:48.920528 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:48.920688 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:48.921260 (+ 572us) server_negotiation.cc:299] Negotiation successful
0811 00:15:48.921508 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 00:15:48.928160 31509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd: Bootstrap starting.
I20260811 00:15:48.932286 31509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd: Neither blocks nor log segments found. Creating new log.
I20260811 00:15:48.933701 31509 log.cc:824] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd: Log is configured to *not* fsync() on all Append() calls
W20260811 00:15:48.936390 31552 tablet.cc:2369] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:15:48.940562 31509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd: No bootstrap required, opened a new log
I20260811 00:15:48.952244 31509 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc2ed543b024be9983a3ef20c1442d5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } } peers { permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } } peers { permanent_uuid: "91b81de49d374083a692a5fcbfb878d0" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } }
I20260811 00:15:48.952652 31509 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:15:48.952768 31509 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6acf6f65cad74985910f3a355fa67bbd, State: Initialized, Role: FOLLOWER
I20260811 00:15:48.953531 31509 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc2ed543b024be9983a3ef20c1442d5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } } peers { permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } } peers { permanent_uuid: "91b81de49d374083a692a5fcbfb878d0" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } }
I20260811 00:15:48.954545 31707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc2ed543b024be9983a3ef20c1442d5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } } peers { permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } } peers { permanent_uuid: "91b81de49d374083a692a5fcbfb878d0" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } } }
I20260811 00:15:48.955027 31707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:15:48.955832 31509 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:15:48.972985 31682 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:15:48.973325 31682 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc2ed543b024be9983a3ef20c1442d5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } } peers { permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } } peers { permanent_uuid: "91b81de49d374083a692a5fcbfb878d0" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } }
I20260811 00:15:48.974915 31682 leader_election.cc:288] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6acf6f65cad74985910f3a355fa67bbd (127.7.94.61:41447), 91b81de49d374083a692a5fcbfb878d0 (127.7.94.60:42723)
I20260811 00:15:48.975978 31582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ebc2ed543b024be9983a3ef20c1442d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b81de49d374083a692a5fcbfb878d0" is_pre_election: true
I20260811 00:15:48.976588 31582 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebc2ed543b024be9983a3ef20c1442d5 in term 0.
I20260811 00:15:48.977201 31289 leader_election.cc:302] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b81de49d374083a692a5fcbfb878d0, ebc2ed543b024be9983a3ef20c1442d5; no voters:
I20260811 00:15:48.978325 31468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ebc2ed543b024be9983a3ef20c1442d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6acf6f65cad74985910f3a355fa67bbd" is_pre_election: true
I20260811 00:15:48.978878 31468 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebc2ed543b024be9983a3ef20c1442d5 in term 0.
W20260811 00:15:48.981317 31728 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:15:48.981500 31728 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:15:48.981546 31402 master_runner.cc:263] No leader master found from master 6acf6f65cad74985910f3a355fa67bbd
I20260811 00:15:48.981678 31402 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 00:15:48.981851 31682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:15:48.982009 31682 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:15:48.982156 31682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:15:48.985599 31682 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc2ed543b024be9983a3ef20c1442d5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } } peers { permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } } peers { permanent_uuid: "91b81de49d374083a692a5fcbfb878d0" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } }
I20260811 00:15:48.986790 31468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ebc2ed543b024be9983a3ef20c1442d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6acf6f65cad74985910f3a355fa67bbd"
I20260811 00:15:48.987010 31468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:15:48.990468 31468 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebc2ed543b024be9983a3ef20c1442d5 in term 1.
I20260811 00:15:48.994122 31289 leader_election.cc:302] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6acf6f65cad74985910f3a355fa67bbd, ebc2ed543b024be9983a3ef20c1442d5; no voters:
I20260811 00:15:48.996723 31682 leader_election.cc:288] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [CANDIDATE]: Term 1 election: Requested vote from peers 6acf6f65cad74985910f3a355fa67bbd (127.7.94.61:41447), 91b81de49d374083a692a5fcbfb878d0 (127.7.94.60:42723)
I20260811 00:15:48.996934 31682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:15:48.997084 31682 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [term 1 LEADER]: Becoming Leader. State: Replica: ebc2ed543b024be9983a3ef20c1442d5, State: Running, Role: LEADER
I20260811 00:15:48.998400 31682 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc2ed543b024be9983a3ef20c1442d5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } } peers { permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } } peers { permanent_uuid: "91b81de49d374083a692a5fcbfb878d0" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } }
I20260811 00:15:49.004706 31582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ebc2ed543b024be9983a3ef20c1442d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b81de49d374083a692a5fcbfb878d0"
I20260811 00:15:49.004948 31582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:15:49.008514 31582 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebc2ed543b024be9983a3ef20c1442d5 in term 1.
I20260811 00:15:49.014233 31730 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebc2ed543b024be9983a3ef20c1442d5. Latest consensus state: current_term: 1 leader_uuid: "ebc2ed543b024be9983a3ef20c1442d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc2ed543b024be9983a3ef20c1442d5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } } peers { permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } } peers { permanent_uuid: "91b81de49d374083a692a5fcbfb878d0" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } } }
I20260811 00:15:49.014708 31730 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [sys.catalog]: This master's current role is: LEADER
I20260811 00:15:49.026480 31735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:15:49.028793 31735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:15:49.044541 31582 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0 [term 1 FOLLOWER]: Refusing update from remote peer ebc2ed543b024be9983a3ef20c1442d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:15:49.045610 31730 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b81de49d374083a692a5fcbfb878d0" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:15:49.052763 31468 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd [term 1 FOLLOWER]: Refusing update from remote peer ebc2ed543b024be9983a3ef20c1442d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:15:49.053591 31730 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:15:49.087538 31707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebc2ed543b024be9983a3ef20c1442d5. Latest consensus state: current_term: 1 leader_uuid: "ebc2ed543b024be9983a3ef20c1442d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc2ed543b024be9983a3ef20c1442d5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } } peers { permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } } peers { permanent_uuid: "91b81de49d374083a692a5fcbfb878d0" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } } }
I20260811 00:15:49.087908 31707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:15:49.092180 31737 mvcc.cc:205] Tried to move back new op lower bound from 7317124501673893888 to 7317124501509214208. Current Snapshot: MvccSnapshot[applied={T|T < 7317124501673893888}]
I20260811 00:15:49.097826 31707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ebc2ed543b024be9983a3ef20c1442d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc2ed543b024be9983a3ef20c1442d5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } } peers { permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } } peers { permanent_uuid: "91b81de49d374083a692a5fcbfb878d0" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } } }
I20260811 00:15:49.098194 31707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:15:49.098801 31740 mvcc.cc:205] Tried to move back new op lower bound from 7317124501673893888 to 7317124501509214208. Current Snapshot: MvccSnapshot[applied={T|T < 7317124501673893888}]
I20260811 00:15:49.105644 31650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebc2ed543b024be9983a3ef20c1442d5. Latest consensus state: current_term: 1 leader_uuid: "ebc2ed543b024be9983a3ef20c1442d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc2ed543b024be9983a3ef20c1442d5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } } peers { permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } } peers { permanent_uuid: "91b81de49d374083a692a5fcbfb878d0" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } } }
I20260811 00:15:49.105978 31650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:15:49.107810 31735 catalog_manager.cc:1347] Generated new cluster ID: c144bbc99873457b81f346701667e1cc
I20260811 00:15:49.107923 31735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:15:49.120425 31682 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ebc2ed543b024be9983a3ef20c1442d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc2ed543b024be9983a3ef20c1442d5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } } peers { permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } } peers { permanent_uuid: "91b81de49d374083a692a5fcbfb878d0" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } } }
I20260811 00:15:49.120764 31682 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [sys.catalog]: This master's current role is: LEADER
I20260811 00:15:49.132830 31735 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:15:49.138577 31730 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ebc2ed543b024be9983a3ef20c1442d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc2ed543b024be9983a3ef20c1442d5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } } peers { permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } } peers { permanent_uuid: "91b81de49d374083a692a5fcbfb878d0" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } } }
I20260811 00:15:49.139043 31730 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5 [sys.catalog]: This master's current role is: LEADER
I20260811 00:15:49.141626 31650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ebc2ed543b024be9983a3ef20c1442d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc2ed543b024be9983a3ef20c1442d5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45191 } } peers { permanent_uuid: "6acf6f65cad74985910f3a355fa67bbd" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 41447 } } peers { permanent_uuid: "91b81de49d374083a692a5fcbfb878d0" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42723 } } }
I20260811 00:15:49.141892 31650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:15:49.142125 31735 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:15:49.160053 31735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebc2ed543b024be9983a3ef20c1442d5: Generated new TSK 0
I20260811 00:15:49.161290 31735 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:15:49.835932 31674 catalog_manager.cc:1259] Loaded cluster ID: c144bbc99873457b81f346701667e1cc
I20260811 00:15:49.836032 31674 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0: loading cluster ID for follower catalog manager: success
I20260811 00:15:49.838574 31674 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0: acquiring CA information for follower catalog manager: success
I20260811 00:15:49.840386 31674 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 91b81de49d374083a692a5fcbfb878d0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 00:15:49.983393 31728 catalog_manager.cc:1259] Loaded cluster ID: c144bbc99873457b81f346701667e1cc
I20260811 00:15:49.983501 31728 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd: loading cluster ID for follower catalog manager: success
I20260811 00:15:49.985888 31728 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd: acquiring CA information for follower catalog manager: success
I20260811 00:15:49.987637 31728 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6acf6f65cad74985910f3a355fa67bbd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 00:15:50.333705 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 31264
I20260811 00:15:50.334813 31378 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:15:50.444190 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 31264
I20260811 00:15:50.481057 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 31402
I20260811 00:15:50.482245 31501 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:15:50.588474 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 31402
I20260811 00:15:50.619378 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 31522
I20260811 00:15:50.620563 31620 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:15:50.728813 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 31522
2026-08-11T00:15:50Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5145 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:15:50 dist-test-slave-8hhm krb5kdc[31779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:15:50 dist-test-slave-8hhm krb5kdc[31779](info): set up 2 sockets
Aug 11 00:15:50 dist-test-slave-8hhm krb5kdc[31779](info): commencing operation
krb5kdc: starting...
W20260811 00:15:52.125139 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.345s user 0.004s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:15:52 dist-test-slave-8hhm krb5kdc[31779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:15:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:15:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:15:52.408277 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:34405
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:34405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:15:52.754827 31797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:15:52.755128 31797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:15:52.755223 31797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:15:52.762398 31797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:15:52.762501 31797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:15:52.762547 31797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:15:52.762605 31797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:15:52.762642 31797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:15:52.772202 31797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:34405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:34405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:15:52.773531 31797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:15:52.774786 31797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:15:52.786584 31812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:52.786716 31816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:52.787077 31809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:52.787622 31797 server_base.cc:1034] running on GCE node
I20260811 00:15:52.788461 31797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:15:52.789885 31797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:15:52.791137 31797 hybrid_clock.cc:648] HybridClock initialized: now 1786407352791094 us; error 49 us; skew 500 ppm
I20260811 00:15:52.791517 31797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:15:52 dist-test-slave-8hhm krb5kdc[31779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407352, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:52.794864 31797 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:15:52.796290 31797 webserver.cc:460] Webserver started at http://127.7.94.62:37485/ using document root <none> and password file <none>
I20260811 00:15:52.796931 31797 fs_manager.cc:359] Metadata directory not provided
I20260811 00:15:52.797026 31797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:15:52.797362 31797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:15:52.800249 31797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "6b45f94c35394580815eacc0de7f7be6"
format_stamp: "Formatted at 2026-08-11 00:15:52 on dist-test-slave-8hhm"
server_key: "cd321310c3a5eea2f6805a44fb724a9c"
server_key_iv: "1400d2a86082bd48ab61807c56163c6d"
server_key_version: "encryptionkey@0"
I20260811 00:15:52.800984 31797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "6b45f94c35394580815eacc0de7f7be6"
format_stamp: "Formatted at 2026-08-11 00:15:52 on dist-test-slave-8hhm"
server_key: "cd321310c3a5eea2f6805a44fb724a9c"
server_key_iv: "1400d2a86082bd48ab61807c56163c6d"
server_key_version: "encryptionkey@0"
I20260811 00:15:52.806115 31797 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 00:15:52.809478 31831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:52.810877 31797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:15:52.811101 31797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "6b45f94c35394580815eacc0de7f7be6"
format_stamp: "Formatted at 2026-08-11 00:15:52 on dist-test-slave-8hhm"
server_key: "cd321310c3a5eea2f6805a44fb724a9c"
server_key_iv: "1400d2a86082bd48ab61807c56163c6d"
server_key_version: "encryptionkey@0"
I20260811 00:15:52.811282 31797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:15:52.842108 31797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:15:52.846742 31797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:15:52.847079 31797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:15:52.866444 31797 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:34405
I20260811 00:15:52.866478 31919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:34405 every 8 connection(s)
I20260811 00:15:52.867871 31797 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:15:52.871820 31920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:15:52.877447 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 31797
I20260811 00:15:52.877688 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:15:52.878332 7544 external_mini_cluster.cc:1428] Setting key e718393ae98fc488dcaa706ed15860b6
I20260811 00:15:52.884227 31920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6: Bootstrap starting.
I20260811 00:15:52.888615 31920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6: Neither blocks nor log segments found. Creating new log.
I20260811 00:15:52.889773 31920 log.cc:824] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6: Log is configured to *not* fsync() on all Append() calls
I20260811 00:15:52.892635 31920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6: No bootstrap required, opened a new log
Aug 11 00:15:52 dist-test-slave-8hhm krb5kdc[31779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:15:52.900417 31920 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b45f94c35394580815eacc0de7f7be6" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34405 } }
I20260811 00:15:52.900823 31920 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:15:52.900934 31920 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b45f94c35394580815eacc0de7f7be6, State: Initialized, Role: FOLLOWER
I20260811 00:15:52.901593 31920 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b45f94c35394580815eacc0de7f7be6" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34405 } }
I20260811 00:15:52.901870 31920 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:15:52.902014 31920 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:15:52.902216 31920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:15:52.904682 31920 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b45f94c35394580815eacc0de7f7be6" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34405 } }
I20260811 00:15:52.905033 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:52.880264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41790 (local address 127.7.94.62:34405)
0811 00:15:52.881144 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:52.881174 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:52.881250 (+ 76us) server_negotiation.cc:407] Connection header received
0811 00:15:52.884554 (+ 3304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:52.884588 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:52.885289 (+ 701us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:52.885782 (+ 493us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:52.888420 (+ 2638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:52.890059 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:52.891639 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:52.892393 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:52.892616 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:52.897179 (+ 4563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:52.897191 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:52.897206 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:52.897317 (+ 111us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:52.900457 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:52.901419 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:52.901426 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:52.901432 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:52.901597 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:52.902065 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:52.902070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:52.902073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:52.902712 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:52.903050 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:52.903550 (+ 500us) server_negotiation.cc:299] Negotiation successful
0811 00:15:52.904005 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":441,"threads_started":1}
I20260811 00:15:52.905318 31920 leader_election.cc:302] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b45f94c35394580815eacc0de7f7be6; no voters:
I20260811 00:15:52.906023 31920 leader_election.cc:288] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:15:52.906149 31927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:15:52.907308 31927 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6 [term 1 LEADER]: Becoming Leader. State: Replica: 6b45f94c35394580815eacc0de7f7be6, State: Running, Role: LEADER
I20260811 00:15:52.907852 31927 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b45f94c35394580815eacc0de7f7be6" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34405 } }
I20260811 00:15:52.908443 31920 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:15:52.909078 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:15:52.910698 31932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b45f94c35394580815eacc0de7f7be6. Latest consensus state: current_term: 1 leader_uuid: "6b45f94c35394580815eacc0de7f7be6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b45f94c35394580815eacc0de7f7be6" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34405 } } }
I20260811 00:15:52.911162 31932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6 [sys.catalog]: This master's current role is: LEADER
I20260811 00:15:52.911826 31931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b45f94c35394580815eacc0de7f7be6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b45f94c35394580815eacc0de7f7be6" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34405 } } }
I20260811 00:15:52.912151 31931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6 [sys.catalog]: This master's current role is: LEADER
I20260811 00:15:52.918865 31940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:15:52.923381 31940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:15:52.931932 31940 catalog_manager.cc:1347] Generated new cluster ID: fa850089ada34ad0bb6c15e0339bc73b
I20260811 00:15:52.932047 31940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:15:52.941203 31940 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:15:52.943156 31940 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:15:52.956697 31940 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b45f94c35394580815eacc0de7f7be6: Generated new TSK 0
I20260811 00:15:52.957770 31940 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:15:52.970180 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 31797
I20260811 00:15:52.971307 31914 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:15:53.061625 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 31797
2026-08-11T00:15:53Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2327 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:15:53 dist-test-slave-8hhm krb5kdc[31983](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:15:53 dist-test-slave-8hhm krb5kdc[31983](info): set up 2 sockets
Aug 11 00:15:53 dist-test-slave-8hhm krb5kdc[31983](info): commencing operation
krb5kdc: starting...
W20260811 00:15:54.420753 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.004s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:15:54 dist-test-slave-8hhm krb5kdc[31983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:15:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:15:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:15:54.715049 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:41023
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:41023
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:15:55.060686 32006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:15:55.061004 32006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:15:55.061169 32006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:15:55.068542 32006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:15:55.068658 32006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:15:55.068706 32006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:15:55.068786 32006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:15:55.068873 32006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:15:55.078843 32006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:41023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:41023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:15:55.080179 32006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:15:55.081579 32006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:15:55.093499 32012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:55.093466 32015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:55.093653 32011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:55.094722 32006 server_base.cc:1034] running on GCE node
I20260811 00:15:55.095527 32006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:15:55.096946 32006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:15:55.098181 32006 hybrid_clock.cc:648] HybridClock initialized: now 1786407355098124 us; error 81 us; skew 500 ppm
I20260811 00:15:55.098534 32006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:15:55 dist-test-slave-8hhm krb5kdc[31983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407355, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:55.101716 32006 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:15:55.103196 32006 webserver.cc:460] Webserver started at http://127.7.94.62:39533/ using document root <none> and password file <none>
I20260811 00:15:55.103960 32006 fs_manager.cc:359] Metadata directory not provided
I20260811 00:15:55.104064 32006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:15:55.104314 32006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:15:55.107399 32006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "24c6f1cb992d4dca8a883053b204fe58"
format_stamp: "Formatted at 2026-08-11 00:15:55 on dist-test-slave-8hhm"
server_key: "e57423cbc46036426f5867c37b27f520"
server_key_iv: "c089d5c51e08afdc459092a8f2668b91"
server_key_version: "encryptionkey@0"
I20260811 00:15:55.108151 32006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "24c6f1cb992d4dca8a883053b204fe58"
format_stamp: "Formatted at 2026-08-11 00:15:55 on dist-test-slave-8hhm"
server_key: "e57423cbc46036426f5867c37b27f520"
server_key_iv: "c089d5c51e08afdc459092a8f2668b91"
server_key_version: "encryptionkey@0"
I20260811 00:15:55.113561 32006 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260811 00:15:55.117100 32032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:55.118515 32006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:15:55.118674 32006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "24c6f1cb992d4dca8a883053b204fe58"
format_stamp: "Formatted at 2026-08-11 00:15:55 on dist-test-slave-8hhm"
server_key: "e57423cbc46036426f5867c37b27f520"
server_key_iv: "c089d5c51e08afdc459092a8f2668b91"
server_key_version: "encryptionkey@0"
I20260811 00:15:55.118846 32006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:15:55.133658 32006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:15:55.137740 32006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:15:55.137980 32006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:15:55.156834 32006 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:41023
I20260811 00:15:55.156852 32127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:41023 every 8 connection(s)
I20260811 00:15:55.158258 32006 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:15:55.161959 32128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:15:55.164873 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 32006
I20260811 00:15:55.165098 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:15:55.165819 7544 external_mini_cluster.cc:1428] Setting key cf5e09e1ee4a1c6845724de9510ddf0a
I20260811 00:15:55.174846 32128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58: Bootstrap starting.
I20260811 00:15:55.178782 32128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58: Neither blocks nor log segments found. Creating new log.
I20260811 00:15:55.180115 32128 log.cc:824] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:15:55 dist-test-slave-8hhm krb5kdc[31983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:15:55.183293 32128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58: No bootstrap required, opened a new log
I20260811 00:15:55.189998 32132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:55.168008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43388 (local address 127.7.94.62:41023)
0811 00:15:55.168828 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:55.168839 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:55.168880 (+ 41us) server_negotiation.cc:407] Connection header received
0811 00:15:55.170555 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:55.170578 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:55.170981 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:55.171326 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:55.173181 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:55.174523 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:55.176410 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:55.177072 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:55.177267 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:55.182396 (+ 5129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:55.182403 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:55.182417 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:55.182454 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:55.185778 (+ 3324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:55.186517 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:55.186523 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:55.186529 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:55.186707 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:55.187100 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:55.187105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:55.187108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:55.187657 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:55.187991 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:55.188454 (+ 463us) server_negotiation.cc:299] Negotiation successful
0811 00:15:55.188944 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":380,"threads_started":1}
I20260811 00:15:55.191601 32128 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c6f1cb992d4dca8a883053b204fe58" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41023 } }
I20260811 00:15:55.192006 32128 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:15:55.192111 32128 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6f1cb992d4dca8a883053b204fe58, State: Initialized, Role: FOLLOWER
I20260811 00:15:55.192703 32128 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c6f1cb992d4dca8a883053b204fe58" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41023 } }
I20260811 00:15:55.192991 32128 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:15:55.193138 32128 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:15:55.193357 32128 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:15:55.195451 32128 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c6f1cb992d4dca8a883053b204fe58" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41023 } }
I20260811 00:15:55.195880 32128 leader_election.cc:302] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24c6f1cb992d4dca8a883053b204fe58; no voters:
I20260811 00:15:55.196458 32128 leader_election.cc:288] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:15:55.196609 32134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:15:55.197826 32134 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 1 LEADER]: Becoming Leader. State: Replica: 24c6f1cb992d4dca8a883053b204fe58, State: Running, Role: LEADER
I20260811 00:15:55.198477 32134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c6f1cb992d4dca8a883053b204fe58" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41023 } }
I20260811 00:15:55.198778 32128 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:15:55.201929 32136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24c6f1cb992d4dca8a883053b204fe58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c6f1cb992d4dca8a883053b204fe58" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41023 } } }
I20260811 00:15:55.202674 32136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [sys.catalog]: This master's current role is: LEADER
I20260811 00:15:55.202482 32138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24c6f1cb992d4dca8a883053b204fe58. Latest consensus state: current_term: 1 leader_uuid: "24c6f1cb992d4dca8a883053b204fe58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c6f1cb992d4dca8a883053b204fe58" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41023 } } }
I20260811 00:15:55.203176 32138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [sys.catalog]: This master's current role is: LEADER
I20260811 00:15:55.204710 32154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:15:55.210067 32154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:15:55.219122 32154 catalog_manager.cc:1347] Generated new cluster ID: 4c4613f49c3a4b0faf5282c9546c16b9
I20260811 00:15:55.219240 32154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:15:55.234568 32154 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:15:55.236508 32154 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:15:55.252519 32154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58: Generated new TSK 0
I20260811 00:15:55.253669 32154 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
I20260811 00:15:55.325385 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41023
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:15:55.703027 32170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:15:55.703367 32170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:15:55.703612 32170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:15:55.711136 32170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:15:55.711272 32170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:15:55.711440 32170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:15:55.721165 32170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:15:55.722644 32170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:15:55.724129 32170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:15:55.737677 32184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:55.737797 32186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:55.737712 32189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:55.738381 32170 server_base.cc:1034] running on GCE node
I20260811 00:15:55.739050 32170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:15:55.740020 32170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:15:55.741272 32170 hybrid_clock.cc:648] HybridClock initialized: now 1786407355741183 us; error 37 us; skew 500 ppm
I20260811 00:15:55.741595 32170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:15:55 dist-test-slave-8hhm krb5kdc[31983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407355, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:55.744705 32170 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:15:55.746176 32170 webserver.cc:460] Webserver started at http://127.7.94.1:35311/ using document root <none> and password file <none>
I20260811 00:15:55.746883 32170 fs_manager.cc:359] Metadata directory not provided
I20260811 00:15:55.747044 32170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:15:55.747371 32170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:15:55.750387 32170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "28832f2904e04b6c86b1b069c2754e46"
format_stamp: "Formatted at 2026-08-11 00:15:55 on dist-test-slave-8hhm"
server_key: "fee595660d09092ee0cd9941e7d1a026"
server_key_iv: "b940d1f63ce0283cf00119124d3fcc8d"
server_key_version: "encryptionkey@0"
I20260811 00:15:55.751219 32170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "28832f2904e04b6c86b1b069c2754e46"
format_stamp: "Formatted at 2026-08-11 00:15:55 on dist-test-slave-8hhm"
server_key: "fee595660d09092ee0cd9941e7d1a026"
server_key_iv: "b940d1f63ce0283cf00119124d3fcc8d"
server_key_version: "encryptionkey@0"
I20260811 00:15:55.756604 32170 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 00:15:55.760116 32201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:55.761463 32170 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:15:55.761641 32170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "28832f2904e04b6c86b1b069c2754e46"
format_stamp: "Formatted at 2026-08-11 00:15:55 on dist-test-slave-8hhm"
server_key: "fee595660d09092ee0cd9941e7d1a026"
server_key_iv: "b940d1f63ce0283cf00119124d3fcc8d"
server_key_version: "encryptionkey@0"
I20260811 00:15:55.761854 32170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:15:55.779690 32170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:15:55.783578 32170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:15:55.783900 32170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:15:55.784878 32170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:15:55.786336 32170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:15:55.786442 32170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:55.786603 32170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:15:55.786677 32170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:55.821374 32170 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:41895
I20260811 00:15:55.821463 32382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:41895 every 8 connection(s)
I20260811 00:15:55.822829 32170 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:15:55.827266 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 32170
I20260811 00:15:55.827498 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:15:55.828167 7544 external_mini_cluster.cc:1428] Setting key d4cfbf4c27232304cae7b36bcdfb8a0c
Aug 11 00:15:55 dist-test-slave-8hhm krb5kdc[31983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407355, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:15:55.848625 32387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:55.825408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:56525 (local address 127.7.94.62:41023)
0811 00:15:55.825882 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:55.825889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:55.826949 (+ 1060us) server_negotiation.cc:407] Connection header received
0811 00:15:55.829023 (+ 2074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:55.829028 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:55.829110 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:55.829294 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:55.831100 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:55.831853 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:55.833487 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:55.833889 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:55.834031 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:55.841746 (+ 7715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:55.841751 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:55.841754 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:55.841787 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:55.845217 (+ 3430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:55.846000 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:55.846008 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:55.846014 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:55.846170 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:55.846584 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:55.846591 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:55.846597 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:55.847016 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:55.847173 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:55.847867 (+ 694us) server_negotiation.cc:299] Negotiation successful
0811 00:15:55.848135 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":230,"threads_started":1}
I20260811 00:15:55.849375 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:55.826034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:56525)
0811 00:15:55.826789 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:55.826829 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:55.828645 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:55.829541 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:55.829553 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:55.830332 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:55.830920 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:55.830938 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:55.832033 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:55.832037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:55.833331 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:55.833349 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:55.834128 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:55.834136 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:55.834392 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:55.838924 (+ 4532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:55.838951 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:55.841478 (+ 2527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:55.845407 (+ 3929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:55.845414 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:55.845427 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:55.845841 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:55.846279 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:55.846282 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:55.846284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:55.846464 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:55.847138 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:55.847145 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:55.847479 (+ 334us) client_negotiation.cc:769] Sending connection context
0811 00:15:55.847704 (+ 225us) client_negotiation.cc:240] Negotiation successful
0811 00:15:55.848102 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":291,"threads_started":1}
I20260811 00:15:55.850973 32385 heartbeater.cc:347] Connected to a master server at 127.7.94.62:41023
I20260811 00:15:55.851346 32385 heartbeater.cc:464] Registering TS with master...
I20260811 00:15:55.852301 32385 heartbeater.cc:511] Master 127.7.94.62:41023 requested a full tablet report, sending...
I20260811 00:15:55.854600 32062 ts_manager.cc:194] Registered new tserver with Master: 28832f2904e04b6c86b1b069c2754e46 (127.7.94.1:41895)
I20260811 00:15:55.856031 32062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:56525
WARNING: no policy specified for kudu/127.7.94.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
I20260811 00:15:55.894757 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41023
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:15:56.238082 32393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:15:56.238359 32393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:15:56.238452 32393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:15:56.245553 32393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:15:56.245657 32393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:15:56.245770 32393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:15:56.255416 32393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:15:56.256706 32393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:15:56.258010 32393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:15:56.271318 32405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:56.271405 32407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:56.271430 32410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:56.272832 32393 server_base.cc:1034] running on GCE node
I20260811 00:15:56.273564 32393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:15:56.274571 32393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:15:56.275779 32393 hybrid_clock.cc:648] HybridClock initialized: now 1786407356275730 us; error 50 us; skew 500 ppm
I20260811 00:15:56.276109 32393 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:15:56 dist-test-slave-8hhm krb5kdc[31983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407356, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:56.279385 32393 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:15:56.280877 32393 webserver.cc:460] Webserver started at http://127.7.94.2:35237/ using document root <none> and password file <none>
I20260811 00:15:56.281672 32393 fs_manager.cc:359] Metadata directory not provided
I20260811 00:15:56.281860 32393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:15:56.282183 32393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:15:56.285086 32393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "d6ea2a8ad06d4a208e6408344df581f2"
format_stamp: "Formatted at 2026-08-11 00:15:56 on dist-test-slave-8hhm"
server_key: "79f91ff0f15822ce41dff5d24d57e153"
server_key_iv: "0a5d2b44968b3821c155cd53e6631d6c"
server_key_version: "encryptionkey@0"
I20260811 00:15:56.285884 32393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "d6ea2a8ad06d4a208e6408344df581f2"
format_stamp: "Formatted at 2026-08-11 00:15:56 on dist-test-slave-8hhm"
server_key: "79f91ff0f15822ce41dff5d24d57e153"
server_key_iv: "0a5d2b44968b3821c155cd53e6631d6c"
server_key_version: "encryptionkey@0"
I20260811 00:15:56.291230 32393 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260811 00:15:56.294780 32419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:56.296205 32393 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 00:15:56.296341 32393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "d6ea2a8ad06d4a208e6408344df581f2"
format_stamp: "Formatted at 2026-08-11 00:15:56 on dist-test-slave-8hhm"
server_key: "79f91ff0f15822ce41dff5d24d57e153"
server_key_iv: "0a5d2b44968b3821c155cd53e6631d6c"
server_key_version: "encryptionkey@0"
I20260811 00:15:56.296499 32393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:15:56.319542 32393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:15:56.323563 32393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:15:56.323812 32393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:15:56.324730 32393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:15:56.326042 32393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:15:56.326120 32393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:56.326205 32393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:15:56.326251 32393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:56.358196 32393 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:42387
I20260811 00:15:56.358247 32606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:42387 every 8 connection(s)
I20260811 00:15:56.359462 32393 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:15:56.363550 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 32393
I20260811 00:15:56.363742 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:15:56.364323 7544 external_mini_cluster.cc:1428] Setting key 53d335dadb7208e46bf5dff8677dcb79
Aug 11 00:15:56 dist-test-slave-8hhm krb5kdc[31983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407356, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:15:56.381963 32611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:56.362141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:40739 (local address 127.7.94.62:41023)
0811 00:15:56.362596 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:56.362600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:56.363825 (+ 1225us) server_negotiation.cc:407] Connection header received
0811 00:15:56.366102 (+ 2277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:56.366108 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:56.366230 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:56.366350 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:56.367846 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:56.369044 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:56.370599 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:56.371361 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:56.371509 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:56.375142 (+ 3633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:56.375151 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:56.375156 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:56.375199 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:56.378601 (+ 3402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:56.379578 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:56.379583 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:56.379585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:56.379721 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:56.380193 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:56.380197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:56.380199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:56.380454 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:56.380609 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:56.381347 (+ 738us) server_negotiation.cc:299] Negotiation successful
0811 00:15:56.381586 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":220,"threads_started":1}
I20260811 00:15:56.382966 32612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:56.362637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.2:40739)
0811 00:15:56.363645 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:56.363699 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:56.365858 (+ 2159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:56.366612 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:56.366621 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:56.367117 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:56.367658 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:56.367679 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:56.369183 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:56.369187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:56.370439 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:56.370451 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:56.371563 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:56.371568 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:56.371785 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:56.372478 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:56.372500 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:56.374909 (+ 2409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:56.378755 (+ 3846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:56.378767 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:56.378787 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:56.379429 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:56.379841 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:56.379846 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:56.379850 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:56.380044 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:56.380583 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:56.380591 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:56.380924 (+ 333us) client_negotiation.cc:769] Sending connection context
0811 00:15:56.381204 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 00:15:56.381799 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":434,"threads_started":1}
I20260811 00:15:56.384487 32607 heartbeater.cc:347] Connected to a master server at 127.7.94.62:41023
I20260811 00:15:56.384864 32607 heartbeater.cc:464] Registering TS with master...
I20260811 00:15:56.385782 32607 heartbeater.cc:511] Master 127.7.94.62:41023 requested a full tablet report, sending...
I20260811 00:15:56.387463 32062 ts_manager.cc:194] Registered new tserver with Master: d6ea2a8ad06d4a208e6408344df581f2 (127.7.94.2:42387)
I20260811 00:15:56.388706 32062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:40739
WARNING: no policy specified for kudu/127.7.94.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
I20260811 00:15:56.435915 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41023
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:15:56.770159 32616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:15:56.770498 32616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:15:56.770675 32616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:15:56.778535 32616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:15:56.778683 32616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:15:56.778932 32616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:15:56.788751 32616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:15:56.790093 32616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:15:56.791457 32616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:15:56.805277 32627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:56.805230 32625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:56.805292 32623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:56.806814 32616 server_base.cc:1034] running on GCE node
I20260811 00:15:56.807539 32616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:15:56.808494 32616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:15:56.809722 32616 hybrid_clock.cc:648] HybridClock initialized: now 1786407356809668 us; error 50 us; skew 500 ppm
I20260811 00:15:56.810046 32616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:15:56 dist-test-slave-8hhm krb5kdc[31983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407356, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:56.813306 32616 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:15:56.814738 32616 webserver.cc:460] Webserver started at http://127.7.94.3:42085/ using document root <none> and password file <none>
I20260811 00:15:56.815450 32616 fs_manager.cc:359] Metadata directory not provided
I20260811 00:15:56.815594 32616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:15:56.815896 32616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:15:56.818893 32616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "a432988472e44d3da84446f55752ecf4"
format_stamp: "Formatted at 2026-08-11 00:15:56 on dist-test-slave-8hhm"
server_key: "4057a7c25208bb83a5f9eae1dbcc41a1"
server_key_iv: "14e151daa9e06e5102acf2f6eae1f7eb"
server_key_version: "encryptionkey@0"
I20260811 00:15:56.819676 32616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "a432988472e44d3da84446f55752ecf4"
format_stamp: "Formatted at 2026-08-11 00:15:56 on dist-test-slave-8hhm"
server_key: "4057a7c25208bb83a5f9eae1dbcc41a1"
server_key_iv: "14e151daa9e06e5102acf2f6eae1f7eb"
server_key_version: "encryptionkey@0"
I20260811 00:15:56.826315 32616 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 00:15:56.830049 32643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:56.831321 32616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:15:56.831488 32616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "a432988472e44d3da84446f55752ecf4"
format_stamp: "Formatted at 2026-08-11 00:15:56 on dist-test-slave-8hhm"
server_key: "4057a7c25208bb83a5f9eae1dbcc41a1"
server_key_iv: "14e151daa9e06e5102acf2f6eae1f7eb"
server_key_version: "encryptionkey@0"
I20260811 00:15:56.831691 32616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:15:56.859639 32385 heartbeater.cc:503] Master 127.7.94.62:41023 was elected leader, sending a full tablet report...
I20260811 00:15:56.859944 32616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:15:56.864627 32616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:15:56.864974 32616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:15:56.866119 32616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:15:56.867527 32616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:15:56.867630 32616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:56.867785 32616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:15:56.867856 32616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:56.913302 32616 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:38383
I20260811 00:15:56.913362 355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:38383 every 8 connection(s)
I20260811 00:15:56.914877 32616 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:15:56.925201 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 32616
I20260811 00:15:56.925465 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:15:56.925902 7544 external_mini_cluster.cc:1428] Setting key 6a7d8de8782291a98fd3c0cbf1e66b8b
Aug 11 00:15:56 dist-test-slave-8hhm krb5kdc[31983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407356, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:15:56.943188 361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:56.917946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:56627 (local address 127.7.94.62:41023)
0811 00:15:56.925647 (+ 7701us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:56.925655 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:56.925678 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:15:56.927861 (+ 2183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:56.927870 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:56.927976 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:56.928110 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:56.930748 (+ 2638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:56.931834 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:56.933520 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:56.933902 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:56.933996 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:56.937756 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:56.937761 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:56.937764 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:56.937796 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:56.940399 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:56.941290 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:56.941294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:56.941296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:56.941414 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:56.941811 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:56.941814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:56.941816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:56.942058 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:56.942164 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:56.942765 (+ 601us) server_negotiation.cc:299] Negotiation successful
0811 00:15:56.942934 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7546,"thread_start_us":7396,"threads_started":1}
I20260811 00:15:56.944124 362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:56.918736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.3:56627)
0811 00:15:56.924513 (+ 5777us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:56.924569 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:56.927367 (+ 2798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:56.928633 (+ 1266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:56.928651 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:56.929628 (+ 977us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:56.930527 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:56.930557 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:56.932016 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:56.932019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:56.933394 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:56.933408 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:56.934163 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:56.934171 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:56.934425 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:56.935116 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:56.935145 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:56.937473 (+ 2328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:56.940541 (+ 3068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:56.940549 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:56.940564 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:56.941124 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:56.941511 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:56.941514 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:56.941516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:56.941711 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:56.942162 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:56.942167 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:56.942447 (+ 280us) client_negotiation.cc:769] Sending connection context
0811 00:15:56.942672 (+ 225us) client_negotiation.cc:240] Negotiation successful
0811 00:15:56.943032 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5475,"thread_start_us":5257,"threads_started":1}
I20260811 00:15:56.945451 357 heartbeater.cc:347] Connected to a master server at 127.7.94.62:41023
I20260811 00:15:56.945711 357 heartbeater.cc:464] Registering TS with master...
I20260811 00:15:56.946372 357 heartbeater.cc:511] Master 127.7.94.62:41023 requested a full tablet report, sending...
I20260811 00:15:56.947659 32062 ts_manager.cc:194] Registered new tserver with Master: a432988472e44d3da84446f55752ecf4 (127.7.94.3:38383)
I20260811 00:15:56.948369 32062 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:56627
I20260811 00:15:56.952963 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:15:56.969702 361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:56.957430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43394 (local address 127.7.94.62:41023)
0811 00:15:56.957619 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:56.957623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:56.958110 (+ 487us) server_negotiation.cc:407] Connection header received
0811 00:15:56.958240 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:56.958243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:56.958310 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:56.958383 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:56.959789 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:56.960499 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:56.961786 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:56.962137 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:56.962255 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:56.964442 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:56.964446 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:56.964449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:56.964479 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:56.967090 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:56.967740 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:56.967746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:56.967749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:56.967872 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:56.968237 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:56.968242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:56.968246 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:56.968549 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:56.968796 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:56.969122 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 00:15:56.969387 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 00:15:56.973752 32062 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:15:56.980546 32062 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:15:57.004483 382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:56.992597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38442 (local address 127.7.94.3:38383)
0811 00:15:56.993468 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:56.993474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:56.994107 (+ 633us) server_negotiation.cc:407] Connection header received
0811 00:15:56.994386 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:56.994399 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:56.994726 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:56.994963 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:15:56.996173 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:56.998162 (+ 1989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.001996 (+ 3834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.003163 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.003293 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:57.003544 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:57.003953 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 00:15:57.004125 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":719,"thread_start_us":550,"threads_started":1}
I20260811 00:15:57.004774 377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:56.992220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:38383 (local address 127.0.0.1:38442)
0811 00:15:56.993284 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:56.993324 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:56.994136 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:56.995340 (+ 1204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:56.995358 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:56.995400 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:15:56.995899 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:56.995916 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:56.999142 (+ 3226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:56.999148 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:57.001812 (+ 2664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:57.001825 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.003400 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.003411 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:57.003664 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:57.003741 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 00:15:57.003885 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 00:15:57.004070 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":1,"thread_start_us":306,"threads_started":1}
I20260811 00:15:57.007870 383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:56.993331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.7.94.1:41895)
0811 00:15:56.993998 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:56.994008 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:56.998322 (+ 4314us) server_negotiation.cc:407] Connection header received
0811 00:15:56.998433 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:56.998440 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:56.998694 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:56.998894 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:15:57.000279 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.002087 (+ 1808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.005065 (+ 2978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.006608 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.006712 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:57.006946 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:57.007501 (+ 555us) server_negotiation.cc:299] Negotiation successful
0811 00:15:57.007650 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":332,"threads_started":1}
I20260811 00:15:57.007952 384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:56.992874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:41895 (local address 127.0.0.1:55776)
0811 00:15:56.997773 (+ 4899us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:56.997791 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:56.997909 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:56.999292 (+ 1383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:56.999297 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:56.999330 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:15:57.000036 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:57.000049 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.002272 (+ 2223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.002277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:57.004864 (+ 2587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:57.004882 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.007121 (+ 2239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.007128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:57.007318 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:57.007350 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 00:15:57.007453 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 00:15:57.007579 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4763,"mutex_wait_us":99,"thread_start_us":4631,"threads_started":1}
I20260811 00:15:57.009367 32729 tablet_service.cc:1467] Processing CreateTablet for tablet 08e6affdc4c7496794d1638bd8e5a338 (DEFAULT_TABLE table=test-table [id=9f6cd6821538483ea49d917d9aa088c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:15:57.010819 32729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08e6affdc4c7496794d1638bd8e5a338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:15:57.011348 32273 tablet_service.cc:1467] Processing CreateTablet for tablet 08e6affdc4c7496794d1638bd8e5a338 (DEFAULT_TABLE table=test-table [id=9f6cd6821538483ea49d917d9aa088c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:15:57.012449 32273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08e6affdc4c7496794d1638bd8e5a338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:15:57.015305 388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:56.994176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55188 (local address 127.7.94.2:42387)
0811 00:15:56.998856 (+ 4680us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:56.998866 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:56.998894 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:15:56.998991 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:56.999000 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:56.999231 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:56.999481 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:15:57.000532 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.002127 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.005180 (+ 3053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.013873 (+ 8693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.014013 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:57.014214 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:57.014538 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 00:15:57.014948 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4538,"thread_start_us":4473,"threads_started":1}
I20260811 00:15:57.015887 387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:56.993978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:42387 (local address 127.0.0.1:55188)
0811 00:15:56.998201 (+ 4223us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:56.998222 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:56.998356 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:56.999476 (+ 1120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:56.999482 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:56.999517 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:15:57.000330 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:57.000345 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.002274 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.002278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:57.005018 (+ 2740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:57.005033 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.014097 (+ 9064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.014104 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:57.014285 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:57.014364 (+ 79us) client_negotiation.cc:769] Sending connection context
0811 00:15:57.014472 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 00:15:57.014629 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4060,"mutex_wait_us":399,"thread_start_us":4002,"threads_started":1}
I20260811 00:15:57.019629 32501 tablet_service.cc:1467] Processing CreateTablet for tablet 08e6affdc4c7496794d1638bd8e5a338 (DEFAULT_TABLE table=test-table [id=9f6cd6821538483ea49d917d9aa088c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:15:57.020925 32501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08e6affdc4c7496794d1638bd8e5a338. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:15:57.024482 389 tablet_bootstrap.cc:492] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46: Bootstrap starting.
I20260811 00:15:57.028134 389 tablet_bootstrap.cc:654] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46: Neither blocks nor log segments found. Creating new log.
I20260811 00:15:57.029592 389 log.cc:824] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46: Log is configured to *not* fsync() on all Append() calls
I20260811 00:15:57.032150 389 tablet_bootstrap.cc:492] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46: No bootstrap required, opened a new log
I20260811 00:15:57.032414 389 ts_tablet_manager.cc:1397] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 00:15:57.033996 392 tablet_bootstrap.cc:492] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4: Bootstrap starting.
I20260811 00:15:57.037879 392 tablet_bootstrap.cc:654] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4: Neither blocks nor log segments found. Creating new log.
I20260811 00:15:57.039238 392 log.cc:824] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4: Log is configured to *not* fsync() on all Append() calls
I20260811 00:15:57.040117 394 tablet_bootstrap.cc:492] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2: Bootstrap starting.
I20260811 00:15:57.043440 394 tablet_bootstrap.cc:654] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2: Neither blocks nor log segments found. Creating new log.
I20260811 00:15:57.043545 389 raft_consensus.cc:348] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:15:57.044015 389 raft_consensus.cc:374] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:15:57.044056 392 tablet_bootstrap.cc:492] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4: No bootstrap required, opened a new log
I20260811 00:15:57.044139 389 raft_consensus.cc:729] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28832f2904e04b6c86b1b069c2754e46, State: Initialized, Role: FOLLOWER
I20260811 00:15:57.044332 392 ts_tablet_manager.cc:1397] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 00:15:57.044642 394 log.cc:824] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2: Log is configured to *not* fsync() on all Append() calls
I20260811 00:15:57.044906 389 consensus_queue.cc:260] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:15:57.046751 389 ts_tablet_manager.cc:1428] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260811 00:15:57.047216 394 tablet_bootstrap.cc:492] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2: No bootstrap required, opened a new log
I20260811 00:15:57.047473 394 ts_tablet_manager.cc:1397] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 00:15:57.054641 392 raft_consensus.cc:348] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:15:57.055110 392 raft_consensus.cc:374] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:15:57.055287 392 raft_consensus.cc:729] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a432988472e44d3da84446f55752ecf4, State: Initialized, Role: FOLLOWER
I20260811 00:15:57.055847 392 consensus_queue.cc:260] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:15:57.056416 394 raft_consensus.cc:348] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:15:57.056762 394 raft_consensus.cc:374] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:15:57.056907 394 raft_consensus.cc:729] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6ea2a8ad06d4a208e6408344df581f2, State: Initialized, Role: FOLLOWER
I20260811 00:15:57.056931 357 heartbeater.cc:503] Master 127.7.94.62:41023 was elected leader, sending a full tablet report...
I20260811 00:15:57.057632 392 ts_tablet_manager.cc:1428] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260811 00:15:57.057550 394 consensus_queue.cc:260] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:15:57.058662 32607 heartbeater.cc:503] Master 127.7.94.62:41023 was elected leader, sending a full tablet report...
I20260811 00:15:57.059283 394 ts_tablet_manager.cc:1428] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
W20260811 00:15:57.075341 32386 tablet.cc:2369] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:15:57.098214 397 raft_consensus.cc:482] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:15:57.098489 397 raft_consensus.cc:504] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:15:57.100358 397 leader_election.cc:288] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a432988472e44d3da84446f55752ecf4 (127.7.94.3:38383), d6ea2a8ad06d4a208e6408344df581f2 (127.7.94.2:42387)
I20260811 00:15:57.106840 382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:57.100853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:49549 (local address 127.7.94.3:38383)
0811 00:15:57.101153 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:57.101160 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:57.101568 (+ 408us) server_negotiation.cc:407] Connection header received
0811 00:15:57.101825 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:57.101832 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:57.101923 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:57.102018 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:15:57.102571 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.103307 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.105109 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.105924 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.106073 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:57.106181 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:57.106461 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 00:15:57.106581 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:15:57.106964 401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:57.100744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:38383 (local address 127.7.94.1:49549)
0811 00:15:57.101451 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:57.101467 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:57.101681 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:57.102050 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:57.102057 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:57.102089 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:15:57.102400 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:57.102408 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.103454 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.103461 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:57.104954 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:57.104967 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.106136 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.106164 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:57.106305 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:57.106328 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 00:15:57.106433 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 00:15:57.106555 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":3,"thread_start_us":390,"threads_started":1}
I20260811 00:15:57.107697 402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:57.101585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:42387 (local address 127.7.94.1:44753)
0811 00:15:57.102239 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:57.102251 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:57.102405 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:57.102749 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:57.102754 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:57.102777 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:15:57.103123 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:57.103131 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.104253 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.104258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:57.105813 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:57.105825 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.106979 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.106985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:57.107163 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:57.107196 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 00:15:57.107339 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 00:15:57.107423 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":320,"threads_started":1}
I20260811 00:15:57.107741 388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:57.101703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:44753 (local address 127.7.94.2:42387)
0811 00:15:57.101942 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:57.101948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:57.102337 (+ 389us) server_negotiation.cc:407] Connection header received
0811 00:15:57.102550 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:57.102562 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:57.102622 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:57.102721 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:15:57.103283 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.104115 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.105963 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.106746 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.106898 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:57.106999 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:57.107374 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 00:15:57.107508 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 00:15:57.108156 32759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08e6affdc4c7496794d1638bd8e5a338" candidate_uuid: "28832f2904e04b6c86b1b069c2754e46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a432988472e44d3da84446f55752ecf4" is_pre_election: true
I20260811 00:15:57.108697 32759 raft_consensus.cc:2454] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28832f2904e04b6c86b1b069c2754e46 in term 0.
I20260811 00:15:57.108712 32536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08e6affdc4c7496794d1638bd8e5a338" candidate_uuid: "28832f2904e04b6c86b1b069c2754e46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6ea2a8ad06d4a208e6408344df581f2" is_pre_election: true
I20260811 00:15:57.109259 32536 raft_consensus.cc:2454] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28832f2904e04b6c86b1b069c2754e46 in term 0.
I20260811 00:15:57.109422 32209 leader_election.cc:302] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28832f2904e04b6c86b1b069c2754e46, a432988472e44d3da84446f55752ecf4; no voters:
I20260811 00:15:57.109889 397 raft_consensus.cc:2781] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:15:57.110062 397 raft_consensus.cc:482] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:15:57.110183 397 raft_consensus.cc:3037] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [term 0 FOLLOWER]: Advancing to term 1
W20260811 00:15:57.111512 32609 tablet.cc:2369] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:15:57.112437 397 raft_consensus.cc:504] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:15:57.113019 397 leader_election.cc:288] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 1 election: Requested vote from peers a432988472e44d3da84446f55752ecf4 (127.7.94.3:38383), d6ea2a8ad06d4a208e6408344df581f2 (127.7.94.2:42387)
I20260811 00:15:57.113556 32759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08e6affdc4c7496794d1638bd8e5a338" candidate_uuid: "28832f2904e04b6c86b1b069c2754e46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a432988472e44d3da84446f55752ecf4"
I20260811 00:15:57.113606 32536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08e6affdc4c7496794d1638bd8e5a338" candidate_uuid: "28832f2904e04b6c86b1b069c2754e46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6ea2a8ad06d4a208e6408344df581f2"
I20260811 00:15:57.113790 32536 raft_consensus.cc:3037] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:15:57.113790 32759 raft_consensus.cc:3037] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:15:57.116954 32536 raft_consensus.cc:2454] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28832f2904e04b6c86b1b069c2754e46 in term 1.
I20260811 00:15:57.116961 32759 raft_consensus.cc:2454] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28832f2904e04b6c86b1b069c2754e46 in term 1.
I20260811 00:15:57.117379 32209 leader_election.cc:302] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28832f2904e04b6c86b1b069c2754e46, d6ea2a8ad06d4a208e6408344df581f2; no voters:
I20260811 00:15:57.117642 397 raft_consensus.cc:2781] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:15:57.118095 397 raft_consensus.cc:686] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [term 1 LEADER]: Becoming Leader. State: Replica: 28832f2904e04b6c86b1b069c2754e46, State: Running, Role: LEADER
I20260811 00:15:57.118551 397 consensus_queue.cc:237] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:15:57.122597 32061 catalog_manager.cc:5495] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28832f2904e04b6c86b1b069c2754e46 (127.7.94.1). New cstate: current_term: 1 leader_uuid: "28832f2904e04b6c86b1b069c2754e46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } health_report { overall_health: HEALTHY } } }
I20260811 00:15:57.164517 383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:57.159152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55778 (local address 127.7.94.1:41895)
0811 00:15:57.159376 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:57.159384 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:57.159406 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:15:57.159516 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:57.159520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:57.159596 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:57.159751 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:15:57.160128 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.160847 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.162324 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.162733 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.162866 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:57.163367 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:15:57.163888 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:15:57.164054 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:57.164121 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 00:15:57.164203 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 00:15:57.166744 358 tablet.cc:2369] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:15:57.173754 32536 raft_consensus.cc:1261] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2 [term 1 FOLLOWER]: Refusing update from remote peer 28832f2904e04b6c86b1b069c2754e46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:15:57.173753 32759 raft_consensus.cc:1261] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4 [term 1 FOLLOWER]: Refusing update from remote peer 28832f2904e04b6c86b1b069c2754e46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:15:57.174811 404 consensus_queue.cc:1035] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:15:57.175216 397 consensus_queue.cc:1035] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:15:57.186434 413 mvcc.cc:205] Tried to move back new op lower bound from 7317124534972010496 to 7317124534761095168. Current Snapshot: MvccSnapshot[applied={T|T < 7317124534972010496}]
I20260811 00:15:57.188048 414 mvcc.cc:205] Tried to move back new op lower bound from 7317124534972010496 to 7317124534761095168. Current Snapshot: MvccSnapshot[applied={T|T < 7317124534972010496}]
I20260811 00:15:57.203626 32062 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:15:57.204007 32062 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:15:57.207530 32062 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58: Sending DeleteTablet for 3 replicas of tablet 08e6affdc4c7496794d1638bd8e5a338
I20260811 00:15:57.208803 32273 tablet_service.cc:1514] Processing DeleteTablet for tablet 08e6affdc4c7496794d1638bd8e5a338 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:15:57 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:55776
I20260811 00:15:57.208966 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 32170
I20260811 00:15:57.209096 32729 tablet_service.cc:1514] Processing DeleteTablet for tablet 08e6affdc4c7496794d1638bd8e5a338 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:15:57 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:38442
I20260811 00:15:57.209848 32501 tablet_service.cc:1514] Processing DeleteTablet for tablet 08e6affdc4c7496794d1638bd8e5a338 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:15:57 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:55188
I20260811 00:15:57.217875 428 tablet_replica.cc:330] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2: stopping tablet replica
I20260811 00:15:57.219439 428 raft_consensus.cc:2229] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:15:57.220181 428 raft_consensus.cc:2258] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:15:57.221228 427 tablet_replica.cc:330] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4: stopping tablet replica
I20260811 00:15:57.221695 427 raft_consensus.cc:2229] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:15:57.222240 427 raft_consensus.cc:2258] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:15:57.224192 428 ts_tablet_manager.cc:1905] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:15:57.224757 427 ts_tablet_manager.cc:1905] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:15:57.224838 424 tablet_replica.cc:330] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46: stopping tablet replica
I20260811 00:15:57.225279 424 raft_consensus.cc:2229] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:15:57.225930 424 raft_consensus.cc:2258] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:15:57.231204 428 ts_tablet_manager.cc:1918] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:15:57.231367 428 log.cc:1196] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/08e6affdc4c7496794d1638bd8e5a338
I20260811 00:15:57.232324 427 ts_tablet_manager.cc:1918] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:15:57.232398 428 ts_tablet_manager.cc:1939] T 08e6affdc4c7496794d1638bd8e5a338 P d6ea2a8ad06d4a208e6408344df581f2: Deleting consensus metadata
I20260811 00:15:57.232589 427 log.cc:1196] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/08e6affdc4c7496794d1638bd8e5a338
I20260811 00:15:57.233002 424 ts_tablet_manager.cc:1905] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:15:57.233305 427 ts_tablet_manager.cc:1939] T 08e6affdc4c7496794d1638bd8e5a338 P a432988472e44d3da84446f55752ecf4: Deleting consensus metadata
I20260811 00:15:57.234325 32038 catalog_manager.cc:4841] TS d6ea2a8ad06d4a208e6408344df581f2 (127.7.94.2:42387): tablet 08e6affdc4c7496794d1638bd8e5a338 (table test-table [id=9f6cd6821538483ea49d917d9aa088c2]) successfully deleted
I20260811 00:15:57.235057 32038 catalog_manager.cc:4841] TS a432988472e44d3da84446f55752ecf4 (127.7.94.3:38383): tablet 08e6affdc4c7496794d1638bd8e5a338 (table test-table [id=9f6cd6821538483ea49d917d9aa088c2]) successfully deleted
I20260811 00:15:57.239670 424 ts_tablet_manager.cc:1918] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:15:57.239851 424 log.cc:1196] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/08e6affdc4c7496794d1638bd8e5a338
I20260811 00:15:57.240376 424 ts_tablet_manager.cc:1939] T 08e6affdc4c7496794d1638bd8e5a338 P 28832f2904e04b6c86b1b069c2754e46: Deleting consensus metadata
Aug 11 00:15:57 dist-test-slave-8hhm krb5kdc[31983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:15:57.241840 32034 catalog_manager.cc:4841] TS 28832f2904e04b6c86b1b069c2754e46 (127.7.94.1:41895): tablet 08e6affdc4c7496794d1638bd8e5a338 (table test-table [id=9f6cd6821538483ea49d917d9aa088c2]) successfully deleted
I20260811 00:15:57.247995 383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:57.210821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55792 (local address 127.7.94.1:41895)
0811 00:15:57.211036 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:57.211045 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:57.218723 (+ 7678us) server_negotiation.cc:407] Connection header received
0811 00:15:57.218971 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:57.218979 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:57.219088 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:57.219214 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:57.226385 (+ 7171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.227563 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.229702 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.230369 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.230467 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:57.242100 (+ 11633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:57.242118 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:57.242132 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:57.242172 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:57.245167 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:57.246049 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:57.246055 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:57.246057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:57.246191 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:57.246590 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:57.246593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:57.246595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:57.246940 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:57.247155 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:57.247510 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 00:15:57.247698 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:15:57.248613 32377 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:15:57.348729 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 32170
I20260811 00:15:57.378756 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 32393
Aug 11 00:15:57 dist-test-slave-8hhm krb5kdc[31983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.2@KRBTEST.COM
I20260811 00:15:57.394004 388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:57.379516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55202 (local address 127.7.94.2:42387)
0811 00:15:57.379699 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:57.379706 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:57.379812 (+ 106us) server_negotiation.cc:407] Connection header received
0811 00:15:57.380039 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:57.380043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:57.380105 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:57.380233 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:57.381569 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.382329 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.383541 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.383944 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.384083 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:57.388428 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:57.388440 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:57.388454 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:57.388511 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:57.391207 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:57.391902 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:57.391909 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:57.391911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:57.392036 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:57.392410 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:57.392415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:57.392418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:57.392781 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:57.393114 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:57.393426 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 00:15:57.393639 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 00:15:57.394655 32600 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:15:57.485459 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 32393
I20260811 00:15:57.512936 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 32616
Aug 11 00:15:57 dist-test-slave-8hhm krb5kdc[31983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.3@KRBTEST.COM
I20260811 00:15:57.529495 382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:57.513774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38444 (local address 127.7.94.3:38383)
0811 00:15:57.513932 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:57.513937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:57.513971 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:15:57.514135 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:57.514139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:57.514209 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:57.514299 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:57.515597 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.516304 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.517841 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:57.518222 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:57.518316 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:57.522932 (+ 4616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:57.522947 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:57.522962 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:57.522994 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:57.526476 (+ 3482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:57.527339 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:57.527349 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:57.527353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:57.527511 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:57.528008 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:57.528013 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:57.528016 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:57.528480 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:57.528711 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:57.528979 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 00:15:57.529150 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 00:15:57.530148 351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:15:57.620332 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 32616
I20260811 00:15:57.648890 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 32006
I20260811 00:15:57.650192 32119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:15:57.761672 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 32006
I20260811 00:15:57.792872 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:41023
--webserver_interface=127.7.94.62
--webserver_port=39533
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:41023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:15:58.130407 435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:15:58.130788 435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:15:58.130961 435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:15:58.138384 435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:15:58.138566 435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:15:58.138681 435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:15:58.138780 435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:15:58.138865 435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:15:58.148757 435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:41023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:41023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=39533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:15:58.150146 435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:15:58.151485 435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:15:58.179292 449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:58.179242 444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:58.179270 447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:58.180368 435 server_base.cc:1034] running on GCE node
I20260811 00:15:58.181293 435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:15:58.182917 435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:15:58.184152 435 hybrid_clock.cc:648] HybridClock initialized: now 1786407358184111 us; error 55 us; skew 500 ppm
I20260811 00:15:58.184554 435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:15:58 dist-test-slave-8hhm krb5kdc[31983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:58.188141 435 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:15:58.189903 435 webserver.cc:460] Webserver started at http://127.7.94.62:39533/ using document root <none> and password file <none>
I20260811 00:15:58.190711 435 fs_manager.cc:359] Metadata directory not provided
I20260811 00:15:58.190892 435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:15:58.196910 435 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 00:15:58.200176 459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:58.201745 435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 00:15:58.201929 435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "24c6f1cb992d4dca8a883053b204fe58"
format_stamp: "Formatted at 2026-08-11 00:15:55 on dist-test-slave-8hhm"
server_key: "e57423cbc46036426f5867c37b27f520"
server_key_iv: "c089d5c51e08afdc459092a8f2668b91"
server_key_version: "encryptionkey@0"
I20260811 00:15:58.203089 435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:15:58.224002 435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:15:58.228222 435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:15:58.228513 435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:15:58.247591 435 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:41023
I20260811 00:15:58.247619 550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:41023 every 8 connection(s)
I20260811 00:15:58.249111 435 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:15:58.250365 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 435
I20260811 00:15:58.251226 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:41895
--local_ip_for_outbound_sockets=127.7.94.1
--tserver_master_addrs=127.7.94.62:41023
--webserver_port=35311
--webserver_interface=127.7.94.1
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:15:58.256523 551 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:15:58.259873 551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58: Bootstrap starting.
I20260811 00:15:58.286836 551 log.cc:824] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58: Log is configured to *not* fsync() on all Append() calls
I20260811 00:15:58.302280 551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:15:58.302958 551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58: Bootstrap complete.
I20260811 00:15:58.316879 551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c6f1cb992d4dca8a883053b204fe58" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41023 } }
I20260811 00:15:58.318130 551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6f1cb992d4dca8a883053b204fe58, State: Initialized, Role: FOLLOWER
I20260811 00:15:58.318928 551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c6f1cb992d4dca8a883053b204fe58" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41023 } }
I20260811 00:15:58.319216 551 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:15:58.319350 551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:15:58.319525 551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 1 FOLLOWER]: Advancing to term 2
I20260811 00:15:58.325371 551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c6f1cb992d4dca8a883053b204fe58" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41023 } }
I20260811 00:15:58.326092 551 leader_election.cc:302] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24c6f1cb992d4dca8a883053b204fe58; no voters:
I20260811 00:15:58.326862 551 leader_election.cc:288] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 00:15:58.327008 563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 00:15:58.328445 563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 2 LEADER]: Becoming Leader. State: Replica: 24c6f1cb992d4dca8a883053b204fe58, State: Running, Role: LEADER
I20260811 00:15:58.329384 551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:15:58.330204 563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c6f1cb992d4dca8a883053b204fe58" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41023 } }
I20260811 00:15:58.332974 565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24c6f1cb992d4dca8a883053b204fe58. Latest consensus state: current_term: 2 leader_uuid: "24c6f1cb992d4dca8a883053b204fe58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c6f1cb992d4dca8a883053b204fe58" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41023 } } }
I20260811 00:15:58.333420 565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [sys.catalog]: This master's current role is: LEADER
I20260811 00:15:58.332993 564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "24c6f1cb992d4dca8a883053b204fe58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c6f1cb992d4dca8a883053b204fe58" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41023 } } }
I20260811 00:15:58.334137 564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [sys.catalog]: This master's current role is: LEADER
I20260811 00:15:58.342515 573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:15:58.351239 573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:15:58.352837 573 catalog_manager.cc:1259] Loaded cluster ID: 4c4613f49c3a4b0faf5282c9546c16b9
I20260811 00:15:58.353020 573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:15:58.356745 573 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 00:15:58.375838 573 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 00:15:58.385375 573 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:15:58.387115 573 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58: Loaded TSK: 0
I20260811 00:15:58.388713 573 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 00:15:58.642560 559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:15:58.642840 559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:15:58.642984 559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:15:58.650686 559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:15:58.650803 559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:15:58.650972 559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:15:58.661008 559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:41895
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=35311
--tserver_master_addrs=127.7.94.62:41023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:15:58.662395 559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:15:58.663820 559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:15:58.683012 607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:58.683776 559 server_base.cc:1034] running on GCE node
W20260811 00:15:58.683923 606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:58.684742 613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:58.685746 559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:15:58.687150 559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:15:58.688441 559 hybrid_clock.cc:648] HybridClock initialized: now 1786407358688384 us; error 58 us; skew 500 ppm
I20260811 00:15:58.688908 559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:15:58 dist-test-slave-8hhm krb5kdc[31983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:58.693593 559 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:15:58.695550 559 webserver.cc:460] Webserver started at http://127.7.94.1:35311/ using document root <none> and password file <none>
I20260811 00:15:58.696597 559 fs_manager.cc:359] Metadata directory not provided
I20260811 00:15:58.696781 559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:15:58.702826 559 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260811 00:15:58.706049 627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:58.707616 559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:15:58.707808 559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "28832f2904e04b6c86b1b069c2754e46"
format_stamp: "Formatted at 2026-08-11 00:15:55 on dist-test-slave-8hhm"
server_key: "fee595660d09092ee0cd9941e7d1a026"
server_key_iv: "b940d1f63ce0283cf00119124d3fcc8d"
server_key_version: "encryptionkey@0"
I20260811 00:15:58.708798 559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:15:58.722518 559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:15:58.727124 559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:15:58.727418 559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:15:58.728472 559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:15:58.730147 559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:15:58.730278 559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:58.730410 559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:15:58.730500 559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:58.767596 559 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:41895
I20260811 00:15:58.767691 821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:41895 every 8 connection(s)
I20260811 00:15:58.769132 559 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:15:58.776974 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 559
I20260811 00:15:58.777807 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:42387
--local_ip_for_outbound_sockets=127.7.94.2
--tserver_master_addrs=127.7.94.62:41023
--webserver_port=35237
--webserver_interface=127.7.94.2
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:15:58 dist-test-slave-8hhm krb5kdc[31983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:15:58.795734 827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:58.772170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:36317 (local address 127.7.94.62:41023)
0811 00:15:58.772904 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:58.772931 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:58.773397 (+ 466us) server_negotiation.cc:407] Connection header received
0811 00:15:58.775706 (+ 2309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:58.775719 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:58.776018 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:58.776322 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:58.778474 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:58.779747 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:58.782247 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:58.782871 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:58.783011 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:58.787287 (+ 4276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:58.787295 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:58.787311 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:58.787353 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:58.791050 (+ 3697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:58.791810 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:58.791820 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:58.791828 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:58.792022 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:58.792474 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:58.792479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:58.792483 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:58.792980 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:58.793205 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:58.794028 (+ 823us) server_negotiation.cc:299] Negotiation successful
0811 00:15:58.794428 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":306,"threads_started":1}
I20260811 00:15:58.795728 830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:58.772336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:36317)
0811 00:15:58.773167 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:58.773253 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:58.775194 (+ 1941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:58.776583 (+ 1389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:58.776594 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:58.777415 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:58.778221 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:58.778249 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:58.779924 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:58.779933 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:58.782112 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:58.782130 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:58.783109 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:58.783118 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:58.783438 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:58.784228 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:58.784265 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:58.787003 (+ 2738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:58.791201 (+ 4198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:58.791210 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:58.791224 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:58.791643 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:58.792144 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:58.792149 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:58.792162 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:58.792348 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:58.793141 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:58.793148 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:58.793511 (+ 363us) client_negotiation.cc:769] Sending connection context
0811 00:15:58.793795 (+ 284us) client_negotiation.cc:240] Negotiation successful
0811 00:15:58.794364 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":349,"threads_started":1}
I20260811 00:15:58.799232 823 heartbeater.cc:347] Connected to a master server at 127.7.94.62:41023
I20260811 00:15:58.799657 823 heartbeater.cc:464] Registering TS with master...
I20260811 00:15:58.800604 823 heartbeater.cc:511] Master 127.7.94.62:41023 requested a full tablet report, sending...
I20260811 00:15:58.803391 492 ts_manager.cc:194] Registered new tserver with Master: 28832f2904e04b6c86b1b069c2754e46 (127.7.94.1:41895)
I20260811 00:15:58.805800 492 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:36317
W20260811 00:15:59.130853 831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:15:59.131139 831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:15:59.131235 831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:15:59.138607 831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:15:59.138720 831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:15:59.138831 831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:15:59.148442 831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:42387
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=35237
--tserver_master_addrs=127.7.94.62:41023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:15:59.149828 831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:15:59.151113 831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:15:59.164256 842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:59.164219 845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:59.164299 840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:59.165007 831 server_base.cc:1034] running on GCE node
I20260811 00:15:59.165721 831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:15:59.166667 831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:15:59.167894 831 hybrid_clock.cc:648] HybridClock initialized: now 1786407359167846 us; error 62 us; skew 500 ppm
I20260811 00:15:59.168231 831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:15:59 dist-test-slave-8hhm krb5kdc[31983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407359, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:59.171358 831 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:15:59.172710 831 webserver.cc:460] Webserver started at http://127.7.94.2:35237/ using document root <none> and password file <none>
I20260811 00:15:59.173448 831 fs_manager.cc:359] Metadata directory not provided
I20260811 00:15:59.173559 831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:15:59.178974 831 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 00:15:59.181672 856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:59.182982 831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 00:15:59.183135 831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "d6ea2a8ad06d4a208e6408344df581f2"
format_stamp: "Formatted at 2026-08-11 00:15:56 on dist-test-slave-8hhm"
server_key: "79f91ff0f15822ce41dff5d24d57e153"
server_key_iv: "0a5d2b44968b3821c155cd53e6631d6c"
server_key_version: "encryptionkey@0"
I20260811 00:15:59.184072 831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:15:59.196928 831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:15:59.200562 831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:15:59.200775 831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:15:59.201723 831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:15:59.202924 831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:15:59.203018 831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:59.203094 831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:15:59.203147 831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:59.237026 831 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:42387
I20260811 00:15:59.237120 1035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:42387 every 8 connection(s)
I20260811 00:15:59.238559 831 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:15:59.242208 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 831
I20260811 00:15:59.242949 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:38383
--local_ip_for_outbound_sockets=127.7.94.3
--tserver_master_addrs=127.7.94.62:41023
--webserver_port=42085
--webserver_interface=127.7.94.3
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:15:59 dist-test-slave-8hhm krb5kdc[31983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407359, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:15:59.261729 827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:59.241110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:35297 (local address 127.7.94.62:41023)
0811 00:15:59.241342 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:59.241348 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:59.242735 (+ 1387us) server_negotiation.cc:407] Connection header received
0811 00:15:59.244647 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:59.244653 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:59.244747 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:59.244860 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:59.246447 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.247455 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.250061 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.250733 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.250902 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:59.255598 (+ 4696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:59.255602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:59.255606 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:59.255637 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:59.258564 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:59.259304 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:59.259310 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:59.259313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:59.259444 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:59.259886 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:59.259891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:59.259894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:59.260188 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:59.260364 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:59.261098 (+ 734us) server_negotiation.cc:299] Negotiation successful
0811 00:15:59.261373 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 00:15:59.262759 1041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:59.241719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.2:35297)
0811 00:15:59.242574 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:59.242620 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:59.244378 (+ 1758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:59.245121 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:59.245130 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:59.245697 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:59.246266 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:59.246282 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.247670 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.247678 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:59.249895 (+ 2217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:59.249918 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.251013 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.251023 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:59.251340 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:59.252253 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:59.252289 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:59.255365 (+ 3076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:59.258731 (+ 3366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:59.258741 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:59.258760 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:59.259175 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:59.259557 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:59.259562 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:59.259565 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:59.259763 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:59.260321 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:59.260332 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:59.260705 (+ 373us) client_negotiation.cc:769] Sending connection context
0811 00:15:59.260982 (+ 277us) client_negotiation.cc:240] Negotiation successful
0811 00:15:59.261569 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":347,"threads_started":1}
I20260811 00:15:59.264515 1036 heartbeater.cc:347] Connected to a master server at 127.7.94.62:41023
I20260811 00:15:59.264853 1036 heartbeater.cc:464] Registering TS with master...
I20260811 00:15:59.265625 1036 heartbeater.cc:511] Master 127.7.94.62:41023 requested a full tablet report, sending...
I20260811 00:15:59.267050 492 ts_manager.cc:194] Registered new tserver with Master: d6ea2a8ad06d4a208e6408344df581f2 (127.7.94.2:42387)
I20260811 00:15:59.267870 492 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:35297
W20260811 00:15:59.592192 1042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:15:59.592480 1042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:15:59.592664 1042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:15:59.599917 1042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:15:59.600023 1042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:15:59.600183 1042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:15:59.610028 1042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:38383
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=42085
--tserver_master_addrs=127.7.94.62:41023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:15:59.611370 1042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:15:59.612872 1042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:15:59.626513 1058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:59.626683 1054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:15:59.626709 1056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:15:59.627830 1042 server_base.cc:1034] running on GCE node
I20260811 00:15:59.628441 1042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:15:59.629455 1042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:15:59.630674 1042 hybrid_clock.cc:648] HybridClock initialized: now 1786407359630618 us; error 49 us; skew 500 ppm
I20260811 00:15:59.630986 1042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:15:59 dist-test-slave-8hhm krb5kdc[31983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407359, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:15:59.634194 1042 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:15:59.635550 1042 webserver.cc:460] Webserver started at http://127.7.94.3:42085/ using document root <none> and password file <none>
I20260811 00:15:59.636188 1042 fs_manager.cc:359] Metadata directory not provided
I20260811 00:15:59.636299 1042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:15:59.641520 1042 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:15:59.644164 1076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:59.645560 1042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:15:59.645707 1042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "a432988472e44d3da84446f55752ecf4"
format_stamp: "Formatted at 2026-08-11 00:15:56 on dist-test-slave-8hhm"
server_key: "4057a7c25208bb83a5f9eae1dbcc41a1"
server_key_iv: "14e151daa9e06e5102acf2f6eae1f7eb"
server_key_version: "encryptionkey@0"
I20260811 00:15:59.646726 1042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:15:59.671432 1042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:15:59.675668 1042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:15:59.675949 1042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:15:59.677114 1042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:15:59.678615 1042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:15:59.678741 1042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:59.678876 1042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:15:59.678966 1042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:15:59.712771 1042 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:38383
I20260811 00:15:59.712879 1251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:38383 every 8 connection(s)
I20260811 00:15:59.714365 1042 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:15:59.718770 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 1042
Aug 11 00:15:59 dist-test-slave-8hhm krb5kdc[31983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407359, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:15:59.735493 1256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:59.719715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43400 (local address 127.7.94.62:41023)
0811 00:15:59.720191 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:59.720197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:59.720553 (+ 356us) server_negotiation.cc:407] Connection header received
0811 00:15:59.720700 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:59.720706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:59.720792 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:59.720899 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:59.723012 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.723849 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.725294 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.725872 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.725980 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:59.728713 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:59.728718 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:59.728721 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:59.728749 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:59.732625 (+ 3876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:59.733392 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:59.733399 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:59.733403 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:59.733564 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:59.733988 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:59.733992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:59.733994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:59.734285 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:59.734569 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:59.734896 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 00:15:59.735128 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"spinlock_wait_cycles":61568,"thread_start_us":254,"threads_started":1}
I20260811 00:15:59.736413 827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:59.717031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:59623 (local address 127.7.94.62:41023)
0811 00:15:59.717283 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:59.717302 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:59.718439 (+ 1137us) server_negotiation.cc:407] Connection header received
0811 00:15:59.720880 (+ 2441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:59.720887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:59.721000 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:59.721121 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:15:59.723214 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.724006 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.725654 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.726195 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.726311 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:59.729708 (+ 3397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:15:59.729716 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:15:59.729720 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:15:59.729754 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:15:59.733112 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:59.733975 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:59.733980 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:59.733985 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:59.734110 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:15:59.734556 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:15:59.734560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:15:59.734562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:15:59.734819 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:15:59.734982 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:59.735981 (+ 999us) server_negotiation.cc:299] Negotiation successful
0811 00:15:59.736132 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77,"spinlock_wait_cycles":6912}
I20260811 00:15:59.737505 1255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:59.717560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.3:59623)
0811 00:15:59.718251 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:59.718288 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:59.720542 (+ 2254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:59.721474 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:59.721488 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:59.722311 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:15:59.723026 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:59.723056 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.724162 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.724166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:59.725514 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:59.725530 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.726391 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.726396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:59.726588 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:59.727085 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:15:59.727106 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:15:59.729190 (+ 2084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:15:59.733266 (+ 4076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:59.733274 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:59.733290 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:59.733852 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:59.734227 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:15:59.734232 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:15:59.734235 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:15:59.734425 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:15:59.734967 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:15:59.734975 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:15:59.735325 (+ 350us) client_negotiation.cc:769] Sending connection context
0811 00:15:59.735654 (+ 329us) client_negotiation.cc:240] Negotiation successful
0811 00:15:59.736176 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":269,"threads_started":1}
I20260811 00:15:59.738864 1253 heartbeater.cc:347] Connected to a master server at 127.7.94.62:41023
I20260811 00:15:59.739179 1253 heartbeater.cc:464] Registering TS with master...
I20260811 00:15:59.739868 1253 heartbeater.cc:511] Master 127.7.94.62:41023 requested a full tablet report, sending...
I20260811 00:15:59.741174 492 ts_manager.cc:194] Registered new tserver with Master: a432988472e44d3da84446f55752ecf4 (127.7.94.3:38383)
I20260811 00:15:59.742081 492 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:59623
I20260811 00:15:59.750680 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:15:59.758077 827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:59.751677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43408 (local address 127.7.94.62:41023)
0811 00:15:59.751877 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:59.751885 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:59.752323 (+ 438us) server_negotiation.cc:407] Connection header received
0811 00:15:59.752560 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:59.752563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:59.752687 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:59.752837 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:15:59.753276 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.754235 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.755828 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.756253 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.756392 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:59.756921 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:15:59.757483 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:15:59.757656 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:59.757716 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 00:15:59.757793 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 00:15:59.758968 492 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:43408:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:15:59.766690 492 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:15:59.810081 823 heartbeater.cc:503] Master 127.7.94.62:41023 was elected leader, sending a full tablet report...
I20260811 00:15:59.813558 1277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:59.779283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:41895 (local address 127.0.0.1:55800)
0811 00:15:59.798412 (+ 19129us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:59.798444 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:59.798651 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:59.802442 (+ 3791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:59.802454 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:59.802499 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:15:59.803030 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:59.803049 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.806801 (+ 3752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.806809 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:59.809824 (+ 3015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:59.809836 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.812267 (+ 2431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.812275 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:59.812516 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:59.812576 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 00:15:59.812923 (+ 347us) client_negotiation.cc:240] Negotiation successful
0811 00:15:59.813149 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18914,"thread_start_us":26144,"threads_started":1}
I20260811 00:15:59.813565 1274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:59.779492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55800 (local address 127.7.94.1:41895)
0811 00:15:59.800980 (+ 21488us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:59.800995 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:59.801035 (+ 40us) server_negotiation.cc:407] Connection header received
0811 00:15:59.801173 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:59.801184 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:59.801585 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:59.801847 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:15:59.804135 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.806630 (+ 2495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.809985 (+ 3355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.811506 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.811618 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:59.811913 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:59.813004 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0811 00:15:59.813230 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21240,"thread_start_us":22582,"threads_started":1}
I20260811 00:15:59.816140 1271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:59.778644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:38383 (local address 127.0.0.1:38448)
0811 00:15:59.779753 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:59.779805 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:59.780001 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:59.808861 (+ 28860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:59.808867 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:59.808895 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:15:59.809339 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:59.809357 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.811287 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.811292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:59.813425 (+ 2133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:59.813439 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.815258 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.815264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:59.815424 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:59.815462 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 00:15:59.815580 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 00:15:59.815705 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"mutex_wait_us":6,"thread_start_us":1007,"threads_started":1}
I20260811 00:15:59.816156 1272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:59.778870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38448 (local address 127.7.94.3:38383)
0811 00:15:59.808289 (+ 29419us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:59.808298 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:59.808331 (+ 33us) server_negotiation.cc:407] Connection header received
0811 00:15:59.808448 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:59.808455 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:59.808673 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:59.808822 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:15:59.809613 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.811106 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.813649 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.815033 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.815184 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:59.815406 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:59.815634 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 00:15:59.815794 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29235,"thread_start_us":29073,"threads_started":1}
I20260811 00:15:59.817739 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:59.780568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:42387 (local address 127.0.0.1:55210)
0811 00:15:59.805734 (+ 25166us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:59.805750 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:59.805843 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:59.809146 (+ 3303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:59.809152 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:59.809179 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:15:59.809765 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:59.809784 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.812233 (+ 2449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.812241 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:59.814513 (+ 2272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:59.814526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.816862 (+ 2336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.816872 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:59.817049 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:59.817079 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 00:15:59.817176 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 00:15:59.817329 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25008,"mutex_wait_us":3,"thread_start_us":18196,"threads_started":1}
I20260811 00:15:59.818629 1279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:59.780930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55210 (local address 127.7.94.2:42387)
0811 00:15:59.807183 (+ 26253us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:59.807195 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:59.807227 (+ 32us) server_negotiation.cc:407] Connection header received
0811 00:15:59.807326 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:59.807335 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:59.807635 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:59.809116 (+ 1481us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:15:59.810548 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.812060 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.814683 (+ 2623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.816123 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.816593 (+ 470us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:59.816785 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:59.818257 (+ 1472us) server_negotiation.cc:299] Negotiation successful
0811 00:15:59.818388 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26072,"thread_start_us":26014,"threads_started":1}
I20260811 00:15:59.818014 705 tablet_service.cc:1467] Processing CreateTablet for tablet 6942eaafc371478d831097ba1334df66 (DEFAULT_TABLE table=test-table [id=5fbc39d4a14840dd9124c768241c6fe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:15:59.819603 705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6942eaafc371478d831097ba1334df66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:15:59.819994 1141 tablet_service.cc:1467] Processing CreateTablet for tablet 6942eaafc371478d831097ba1334df66 (DEFAULT_TABLE table=test-table [id=5fbc39d4a14840dd9124c768241c6fe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:15:59.821485 1141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6942eaafc371478d831097ba1334df66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:15:59.821807 929 tablet_service.cc:1467] Processing CreateTablet for tablet 6942eaafc371478d831097ba1334df66 (DEFAULT_TABLE table=test-table [id=5fbc39d4a14840dd9124c768241c6fe3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:15:59.822830 929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6942eaafc371478d831097ba1334df66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:15:59.835047 1280 tablet_bootstrap.cc:492] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2: Bootstrap starting.
I20260811 00:15:59.838033 1282 tablet_bootstrap.cc:492] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4: Bootstrap starting.
I20260811 00:15:59.839247 1280 tablet_bootstrap.cc:654] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2: Neither blocks nor log segments found. Creating new log.
I20260811 00:15:59.840057 1281 tablet_bootstrap.cc:492] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46: Bootstrap starting.
I20260811 00:15:59.841575 1280 log.cc:824] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2: Log is configured to *not* fsync() on all Append() calls
I20260811 00:15:59.841746 1282 tablet_bootstrap.cc:654] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4: Neither blocks nor log segments found. Creating new log.
I20260811 00:15:59.842921 1282 log.cc:824] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4: Log is configured to *not* fsync() on all Append() calls
I20260811 00:15:59.843334 1281 tablet_bootstrap.cc:654] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46: Neither blocks nor log segments found. Creating new log.
I20260811 00:15:59.844584 1281 log.cc:824] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46: Log is configured to *not* fsync() on all Append() calls
I20260811 00:15:59.844952 1280 tablet_bootstrap.cc:492] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2: No bootstrap required, opened a new log
I20260811 00:15:59.845302 1280 ts_tablet_manager.cc:1397] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 00:15:59.846217 1282 tablet_bootstrap.cc:492] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4: No bootstrap required, opened a new log
I20260811 00:15:59.846495 1282 ts_tablet_manager.cc:1397] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 00:15:59.847390 1281 tablet_bootstrap.cc:492] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46: No bootstrap required, opened a new log
I20260811 00:15:59.847700 1281 ts_tablet_manager.cc:1397] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 00:15:59.855454 1281 raft_consensus.cc:348] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:15:59.855844 1281 raft_consensus.cc:374] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:15:59.855985 1281 raft_consensus.cc:729] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28832f2904e04b6c86b1b069c2754e46, State: Initialized, Role: FOLLOWER
I20260811 00:15:59.856606 1281 consensus_queue.cc:260] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:15:59.857402 1282 raft_consensus.cc:348] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:15:59.857402 1280 raft_consensus.cc:348] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:15:59.857832 1280 raft_consensus.cc:374] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:15:59.857832 1282 raft_consensus.cc:374] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:15:59.857924 1282 raft_consensus.cc:729] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a432988472e44d3da84446f55752ecf4, State: Initialized, Role: FOLLOWER
I20260811 00:15:59.857924 1280 raft_consensus.cc:729] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6ea2a8ad06d4a208e6408344df581f2, State: Initialized, Role: FOLLOWER
I20260811 00:15:59.858453 1281 ts_tablet_manager.cc:1428] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260811 00:15:59.858521 1282 consensus_queue.cc:260] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:15:59.858551 1280 consensus_queue.cc:260] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:15:59.860437 1253 heartbeater.cc:503] Master 127.7.94.62:41023 was elected leader, sending a full tablet report...
I20260811 00:15:59.860467 1036 heartbeater.cc:503] Master 127.7.94.62:41023 was elected leader, sending a full tablet report...
I20260811 00:15:59.861037 1282 ts_tablet_manager.cc:1428] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260811 00:15:59.862134 1280 ts_tablet_manager.cc:1428] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2: Time spent starting tablet: real 0.017s user 0.014s sys 0.000s
I20260811 00:15:59.905539 1292 raft_consensus.cc:482] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:15:59.905884 1292 raft_consensus.cc:504] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:15:59.907648 1292 leader_election.cc:288] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6ea2a8ad06d4a208e6408344df581f2 (127.7.94.2:42387), 28832f2904e04b6c86b1b069c2754e46 (127.7.94.1:41895)
I20260811 00:15:59.913424 1279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:59.908238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:56647 (local address 127.7.94.2:42387)
0811 00:15:59.908432 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:59.908438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:59.908460 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:15:59.908662 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:59.908666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:59.908729 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:59.908837 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:15:59.909323 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.910027 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.911592 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.912493 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.912635 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:59.912727 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:59.913008 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 00:15:59.913140 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:15:59.913429 1255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:59.908154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:42387 (local address 127.7.94.3:56647)
0811 00:15:59.908333 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:59.908354 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:59.908484 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:59.908869 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:59.908878 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:59.908929 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:15:59.909127 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:59.909136 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.910166 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.910174 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:59.911426 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:59.911438 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.912666 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.912672 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:59.912853 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:59.912876 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 00:15:59.912979 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 00:15:59.913133 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 00:15:59.914404 957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6942eaafc371478d831097ba1334df66" candidate_uuid: "a432988472e44d3da84446f55752ecf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6ea2a8ad06d4a208e6408344df581f2" is_pre_election: true
I20260811 00:15:59.914583 1274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:59.908224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:59393 (local address 127.7.94.1:41895)
0811 00:15:59.908431 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:59.908438 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:59.909306 (+ 868us) server_negotiation.cc:407] Connection header received
0811 00:15:59.909525 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:59.909535 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:59.909592 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:59.909702 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:15:59.910206 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.910891 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.912632 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.913683 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.913851 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:59.913954 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:59.914229 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 00:15:59.914350 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:15:59.914692 1294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:59.908162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:41895 (local address 127.7.94.3:59393)
0811 00:15:59.909130 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0811 00:15:59.909145 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 00:15:59.909389 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:15:59.909720 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:15:59.909724 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:15:59.909747 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:15:59.910042 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:59.910050 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.911036 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.911044 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:59.912471 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:15:59.912488 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.913890 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.913896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:15:59.914068 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:59.914090 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 00:15:59.914203 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 00:15:59.914323 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":847,"mutex_wait_us":195,"thread_start_us":414,"threads_started":1}
I20260811 00:15:59.914943 957 raft_consensus.cc:2454] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a432988472e44d3da84446f55752ecf4 in term 0.
I20260811 00:15:59.915699 1081 leader_election.cc:302] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a432988472e44d3da84446f55752ecf4, d6ea2a8ad06d4a208e6408344df581f2; no voters:
I20260811 00:15:59.915706 731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6942eaafc371478d831097ba1334df66" candidate_uuid: "a432988472e44d3da84446f55752ecf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28832f2904e04b6c86b1b069c2754e46" is_pre_election: true
I20260811 00:15:59.916137 1292 raft_consensus.cc:2781] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:15:59.916291 1292 raft_consensus.cc:482] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:15:59.916237 731 raft_consensus.cc:2454] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a432988472e44d3da84446f55752ecf4 in term 0.
I20260811 00:15:59.916410 1292 raft_consensus.cc:3037] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:15:59.918757 1292 raft_consensus.cc:504] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:15:59.919332 1292 leader_election.cc:288] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [CANDIDATE]: Term 1 election: Requested vote from peers d6ea2a8ad06d4a208e6408344df581f2 (127.7.94.2:42387), 28832f2904e04b6c86b1b069c2754e46 (127.7.94.1:41895)
I20260811 00:15:59.919771 731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6942eaafc371478d831097ba1334df66" candidate_uuid: "a432988472e44d3da84446f55752ecf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28832f2904e04b6c86b1b069c2754e46"
I20260811 00:15:59.919772 957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6942eaafc371478d831097ba1334df66" candidate_uuid: "a432988472e44d3da84446f55752ecf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6ea2a8ad06d4a208e6408344df581f2"
I20260811 00:15:59.919962 957 raft_consensus.cc:3037] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:15:59.919956 731 raft_consensus.cc:3037] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:15:59.922216 957 raft_consensus.cc:2454] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a432988472e44d3da84446f55752ecf4 in term 1.
I20260811 00:15:59.922228 731 raft_consensus.cc:2454] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a432988472e44d3da84446f55752ecf4 in term 1.
I20260811 00:15:59.922603 1081 leader_election.cc:302] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a432988472e44d3da84446f55752ecf4, d6ea2a8ad06d4a208e6408344df581f2; no voters:
I20260811 00:15:59.922907 1292 raft_consensus.cc:2781] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:15:59.923352 1292 raft_consensus.cc:686] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [term 1 LEADER]: Becoming Leader. State: Replica: a432988472e44d3da84446f55752ecf4, State: Running, Role: LEADER
I20260811 00:15:59.923883 1292 consensus_queue.cc:237] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:15:59.928289 489 catalog_manager.cc:5495] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 reported cstate change: term changed from 0 to 1, leader changed from <none> to a432988472e44d3da84446f55752ecf4 (127.7.94.3). New cstate: current_term: 1 leader_uuid: "a432988472e44d3da84446f55752ecf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } health_report { overall_health: UNKNOWN } } }
W20260811 00:15:59.965986 1254 tablet.cc:2369] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:15:59.966928 1272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:15:59.961610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38450 (local address 127.7.94.3:38383)
0811 00:15:59.961832 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 00:15:59.961837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:15:59.961856 (+ 19us) server_negotiation.cc:407] Connection header received
0811 00:15:59.962020 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:15:59.962024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:15:59.962119 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:15:59.962262 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:15:59.962575 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.963306 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.964629 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:15:59.965010 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:15:59.965107 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:15:59.965631 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:15:59.966378 (+ 747us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:15:59.966527 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 00:15:59.966579 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 00:15:59.966668 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 00:15:59.975049 957 raft_consensus.cc:1261] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2 [term 1 FOLLOWER]: Refusing update from remote peer a432988472e44d3da84446f55752ecf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:15:59.975051 731 raft_consensus.cc:1261] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Refusing update from remote peer a432988472e44d3da84446f55752ecf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:15:59.976007 1292 consensus_queue.cc:1035] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:15:59.976311 1296 consensus_queue.cc:1035] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:15:59.990375 1306 mvcc.cc:205] Tried to move back new op lower bound from 7317124546447855616 to 7317124546252640256. Current Snapshot: MvccSnapshot[applied={T|T < 7317124546447855616}]
I20260811 00:16:00.006273 492 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:43408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:16:00.006567 492 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:43408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:16:00.010162 492 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58: Sending DeleteTablet for 3 replicas of tablet 6942eaafc371478d831097ba1334df66
I20260811 00:16:00.011473 929 tablet_service.cc:1514] Processing DeleteTablet for tablet 6942eaafc371478d831097ba1334df66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:16:00 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:55210
I20260811 00:16:00.011596 1141 tablet_service.cc:1514] Processing DeleteTablet for tablet 6942eaafc371478d831097ba1334df66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:16:00 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:38448
I20260811 00:16:00.011906 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 559
I20260811 00:16:00.012398 705 tablet_service.cc:1514] Processing DeleteTablet for tablet 6942eaafc371478d831097ba1334df66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:16:00 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:55800
I20260811 00:16:00.034611 1323 tablet_replica.cc:330] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46: stopping tablet replica
I20260811 00:16:00.035115 1323 raft_consensus.cc:2229] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:16:00.035674 1323 raft_consensus.cc:2258] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:16:00.036782 1321 tablet_replica.cc:330] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4: stopping tablet replica
I20260811 00:16:00.037163 1274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:00.013116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55808 (local address 127.7.94.1:41895)
0811 00:16:00.013320 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:00.013328 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:00.013350 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:16:00.013415 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:00.013423 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:00.013514 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:00.013630 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:00.015890 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:00.017318 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:00.019765 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:00.023251 (+ 3486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:00.023401 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:00.027744 (+ 4343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:00.027762 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:00.027785 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:00.027835 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:00.032508 (+ 4673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:00.033596 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:00.033609 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:00.033615 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:00.033792 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:00.034316 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:00.034321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:00.034323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:00.034935 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:00.035337 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:00.036028 (+ 691us) server_negotiation.cc:299] Negotiation successful
0811 00:16:00.036690 (+ 662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 00:16:00.037297 1321 raft_consensus.cc:2229] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:16:00.038045 1321 raft_consensus.cc:2258] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:16:00.038926 816 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:16:00.040849 1319 tablet_replica.cc:330] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2: stopping tablet replica
I20260811 00:16:00.041643 1319 raft_consensus.cc:2229] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:16:00.042265 1319 raft_consensus.cc:2258] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:16:00.043429 1321 ts_tablet_manager.cc:1905] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:16:00.044739 1319 ts_tablet_manager.cc:1905] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:16:00.051667 1321 ts_tablet_manager.cc:1918] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:16:00.051959 1321 log.cc:1196] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/6942eaafc371478d831097ba1334df66
I20260811 00:16:00.052642 1321 ts_tablet_manager.cc:1939] T 6942eaafc371478d831097ba1334df66 P a432988472e44d3da84446f55752ecf4: Deleting consensus metadata
I20260811 00:16:00.054723 463 catalog_manager.cc:4841] TS a432988472e44d3da84446f55752ecf4 (127.7.94.3:38383): tablet 6942eaafc371478d831097ba1334df66 (table test-table [id=5fbc39d4a14840dd9124c768241c6fe3]) successfully deleted
I20260811 00:16:00.054800 1319 ts_tablet_manager.cc:1918] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:16:00.054945 1319 log.cc:1196] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/6942eaafc371478d831097ba1334df66
I20260811 00:16:00.055608 1319 ts_tablet_manager.cc:1939] T 6942eaafc371478d831097ba1334df66 P d6ea2a8ad06d4a208e6408344df581f2: Deleting consensus metadata
I20260811 00:16:00.057778 463 catalog_manager.cc:4841] TS d6ea2a8ad06d4a208e6408344df581f2 (127.7.94.2:42387): tablet 6942eaafc371478d831097ba1334df66 (table test-table [id=5fbc39d4a14840dd9124c768241c6fe3]) successfully deleted
I20260811 00:16:00.040000 1323 ts_tablet_manager.cc:1905] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:16:00.156100 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 559
W20260811 00:16:00.182876 460 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.94.1:41895 (error 108)
I20260811 00:16:00.183631 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 831
I20260811 00:16:00.185092 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:00.184523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:41895 (local address 127.0.0.1:55822)
0811 00:16:00.184735 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:00.184879 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:41895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260811 00:16:00.185416 460 catalog_manager.cc:4584] TS 28832f2904e04b6c86b1b069c2754e46 (127.7.94.1:41895): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6942eaafc371478d831097ba1334df66: Network error: Client connection negotiation failed: client connection to 127.7.94.1:41895: connect: Connection refused (error 111)
I20260811 00:16:00.199383 1279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:00.184530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55226 (local address 127.7.94.2:42387)
0811 00:16:00.184791 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:00.184800 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:00.184823 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:16:00.185038 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:00.185043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:00.185138 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:00.185292 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:00.187088 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:00.187901 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:00.189839 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:00.190228 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:00.190334 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:00.193597 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:00.193611 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:00.193625 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:00.193660 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:00.196592 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:00.197422 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:00.197428 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:00.197430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:00.197547 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:00.197960 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:00.197964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:00.197966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:00.198335 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:00.198545 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:00.198902 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 00:16:00.199063 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260811 00:16:00.200194 1029 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:16:00.239331 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:00.238759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:41895 (local address 127.0.0.1:55836)
0811 00:16:00.238992 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:00.239132 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:41895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 00:16:00.240533 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:00.240121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:41895 (local address 127.0.0.1:55838)
0811 00:16:00.240288 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:00.240386 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:41895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 00:16:00.301438 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:00.300695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:41895 (local address 127.0.0.1:55850)
0811 00:16:00.301069 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:00.301199 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:41895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187}
I20260811 00:16:00.302312 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 831
I20260811 00:16:00.303819 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:00.302420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:41895 (local address 127.0.0.1:55862)
0811 00:16:00.303561 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:00.303664 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:41895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1004}
I20260811 00:16:00.333773 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 1042
I20260811 00:16:00.351719 1272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:00.334704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38462 (local address 127.7.94.3:38383)
0811 00:16:00.334879 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:00.334883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:00.334965 (+ 82us) server_negotiation.cc:407] Connection header received
0811 00:16:00.335192 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:00.335196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:00.335253 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:00.335335 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:00.337371 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:00.338222 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:00.340319 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:00.340744 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:00.340832 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:00.344355 (+ 3523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:00.344369 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:00.344385 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:00.344425 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:00.348505 (+ 4080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:00.349354 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:00.349368 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:00.349372 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:00.349566 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:00.349999 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:00.350006 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:00.350015 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:00.350472 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:00.350791 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:00.351025 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 00:16:00.351256 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 00:16:00.352481 1245 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:16:00.384830 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:00.384263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:41895 (local address 127.0.0.1:55864)
0811 00:16:00.384513 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:00.384635 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:41895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260811 00:16:00.385965 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:00.385543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:41895 (local address 127.0.0.1:55872)
0811 00:16:00.385755 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:00.385836 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:41895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260811 00:16:00.477196 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 1042
I20260811 00:16:00.507369 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 435
I20260811 00:16:00.508517 543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:16:00.623668 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:00.622964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:41895 (local address 127.0.0.1:55878)
0811 00:16:00.623318 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:00.623438 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:41895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205}
I20260811 00:16:00.624231 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 435
I20260811 00:16:00.660398 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:41023
--webserver_interface=127.7.94.62
--webserver_port=39533
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:41023
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:16:01.009009 1330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:16:01.009418 1330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:16:01.009584 1330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:16:01.016932 1330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:16:01.017083 1330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:16:01.017196 1330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:16:01.017338 1330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:16:01.017427 1330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:16:01.027413 1330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:41023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:41023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=39533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:16:01.028774 1330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:16:01.030189 1330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:16:01.042310 1335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:01.042286 1336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:01.042238 1338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:16:01.043321 1330 server_base.cc:1034] running on GCE node
I20260811 00:16:01.044113 1330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:16:01.045574 1330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:16:01.046802 1330 hybrid_clock.cc:648] HybridClock initialized: now 1786407361046757 us; error 55 us; skew 500 ppm
I20260811 00:16:01.047158 1330 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:16:01 dist-test-slave-8hhm krb5kdc[31983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407361, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:16:01.055116 1330 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:16:01.056730 1330 webserver.cc:460] Webserver started at http://127.7.94.62:39533/ using document root <none> and password file <none>
I20260811 00:16:01.057530 1330 fs_manager.cc:359] Metadata directory not provided
I20260811 00:16:01.057691 1330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:16:01.063076 1330 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.004s
I20260811 00:16:01.065825 1353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:01.067026 1330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:16:01.067212 1330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "24c6f1cb992d4dca8a883053b204fe58"
format_stamp: "Formatted at 2026-08-11 00:15:55 on dist-test-slave-8hhm"
server_key: "e57423cbc46036426f5867c37b27f520"
server_key_iv: "c089d5c51e08afdc459092a8f2668b91"
server_key_version: "encryptionkey@0"
I20260811 00:16:01.068184 1330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:16:01.093678 1330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:16:01.098174 1330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:16:01.098534 1330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:16:01.119635 1330 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:41023
I20260811 00:16:01.119654 1440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:41023 every 8 connection(s)
I20260811 00:16:01.121405 1330 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:16:01.128209 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 1330
I20260811 00:16:01.128336 1441 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:16:01.129123 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:41895
--local_ip_for_outbound_sockets=127.7.94.1
--tserver_master_addrs=127.7.94.62:41023
--webserver_port=35311
--webserver_interface=127.7.94.1
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:16:01.131668 1441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58: Bootstrap starting.
I20260811 00:16:01.152050 1441 log.cc:824] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58: Log is configured to *not* fsync() on all Append() calls
I20260811 00:16:01.168206 1441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:16:01.168771 1441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58: Bootstrap complete.
I20260811 00:16:01.178632 1441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c6f1cb992d4dca8a883053b204fe58" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41023 } }
I20260811 00:16:01.179504 1441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c6f1cb992d4dca8a883053b204fe58, State: Initialized, Role: FOLLOWER
I20260811 00:16:01.180200 1441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c6f1cb992d4dca8a883053b204fe58" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41023 } }
I20260811 00:16:01.180429 1441 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:16:01.180578 1441 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:16:01.180794 1441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 2 FOLLOWER]: Advancing to term 3
I20260811 00:16:01.184804 1441 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c6f1cb992d4dca8a883053b204fe58" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41023 } }
I20260811 00:16:01.185417 1441 leader_election.cc:302] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24c6f1cb992d4dca8a883053b204fe58; no voters:
I20260811 00:16:01.186127 1441 leader_election.cc:288] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 00:16:01.186267 1450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 00:16:01.187732 1450 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [term 3 LEADER]: Becoming Leader. State: Replica: 24c6f1cb992d4dca8a883053b204fe58, State: Running, Role: LEADER
I20260811 00:16:01.188339 1450 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c6f1cb992d4dca8a883053b204fe58" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41023 } }
I20260811 00:16:01.188565 1441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:16:01.191037 1452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24c6f1cb992d4dca8a883053b204fe58. Latest consensus state: current_term: 3 leader_uuid: "24c6f1cb992d4dca8a883053b204fe58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c6f1cb992d4dca8a883053b204fe58" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41023 } } }
I20260811 00:16:01.191716 1452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [sys.catalog]: This master's current role is: LEADER
I20260811 00:16:01.191563 1451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "24c6f1cb992d4dca8a883053b204fe58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24c6f1cb992d4dca8a883053b204fe58" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41023 } } }
I20260811 00:16:01.192039 1451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24c6f1cb992d4dca8a883053b204fe58 [sys.catalog]: This master's current role is: LEADER
I20260811 00:16:01.194442 1459 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:16:01.205081 1459 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:16:01.207165 1459 catalog_manager.cc:1259] Loaded cluster ID: 4c4613f49c3a4b0faf5282c9546c16b9
I20260811 00:16:01.207295 1459 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 00:16:01.209996 1459 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f004e4959ad google::LogMessage::Fail() at ??:0
@ 0x7f004e49888a google::LogMessage::SendToLog() at ??:0
@ 0x7f004e4954de google::LogMessage::Flush() at ??:0
@ 0x7f004e496a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0063ecc512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f0063ec9976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0063f51610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f0063f51439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f00510e1345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f00510e491c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f00510e4799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f00510bfeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f00543996db start_thread at ??:0
@ 0x7f004d32f61f clone at ??:0
W20260811 00:16:01.552673 1448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:16:01.553005 1448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:16:01.553203 1448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:16:01.560732 1448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:16:01.560853 1448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:16:01.561074 1448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:16:01.571010 1448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:41895
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=35311
--tserver_master_addrs=127.7.94.62:41023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:16:01.572497 1448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:16:01.574034 1448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:16:01.587926 1513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:01.588008 1509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:01.588047 1510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:16:01.588980 1448 server_base.cc:1034] running on GCE node
I20260811 00:16:01.589674 1448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:16:01.590636 1448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:16:01.591867 1448 hybrid_clock.cc:648] HybridClock initialized: now 1786407361591818 us; error 53 us; skew 500 ppm
I20260811 00:16:01.592226 1448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:16:01 dist-test-slave-8hhm krb5kdc[31983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407361, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:16:01.595381 1448 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:16:01.596735 1448 webserver.cc:460] Webserver started at http://127.7.94.1:35311/ using document root <none> and password file <none>
I20260811 00:16:01.597553 1448 fs_manager.cc:359] Metadata directory not provided
I20260811 00:16:01.597726 1448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:16:01.602943 1448 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 00:16:01.608096 1527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:01.609757 1448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:16:01.609961 1448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "28832f2904e04b6c86b1b069c2754e46"
format_stamp: "Formatted at 2026-08-11 00:15:55 on dist-test-slave-8hhm"
server_key: "fee595660d09092ee0cd9941e7d1a026"
server_key_iv: "b940d1f63ce0283cf00119124d3fcc8d"
server_key_version: "encryptionkey@0"
I20260811 00:16:01.611042 1448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:16:01.624415 1448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:16:01.628717 1448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:16:01.629045 1448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:16:01.630136 1448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:16:01.632092 1541 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 00:16:01.637770 1448 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 00:16:01.637919 1448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260811 00:16:01.638083 1448 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 00:16:01.641436 1448 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 00:16:01.641530 1448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.004s sys 0.000s
I20260811 00:16:01.641875 1541 tablet_bootstrap.cc:492] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46: Bootstrap starting.
I20260811 00:16:01.649922 1541 log.cc:824] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46: Log is configured to *not* fsync() on all Append() calls
I20260811 00:16:01.664916 1541 tablet_bootstrap.cc:492] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:16:01.665750 1541 tablet_bootstrap.cc:492] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46: Bootstrap complete.
I20260811 00:16:01.666374 1541 ts_tablet_manager.cc:1397] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46: Time spent bootstrapping tablet: real 0.025s user 0.009s sys 0.014s
I20260811 00:16:01.675621 1541 raft_consensus.cc:348] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:16:01.677340 1541 raft_consensus.cc:729] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28832f2904e04b6c86b1b069c2754e46, State: Initialized, Role: FOLLOWER
I20260811 00:16:01.678318 1541 consensus_queue.cc:260] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:16:01.680387 1541 ts_tablet_manager.cc:1428] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46: Time spent starting tablet: real 0.014s user 0.008s sys 0.006s
I20260811 00:16:01.686587 1448 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:41895
I20260811 00:16:01.686666 1733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:41895 every 8 connection(s)
I20260811 00:16:01.687879 1448 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:16:01.692552 1736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:01.690688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:34345)
0811 00:16:01.691490 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:01.691679 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":355,"threads_started":1}
W20260811 00:16:01.693372 1734 heartbeater.cc:644] Failed to heartbeat to 127.7.94.62:41023 (0 consecutive failures): Network error: Failed to ping master at 127.7.94.62:41023: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
I20260811 00:16:01.694221 1736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:01.693869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:48509)
0811 00:16:01.694015 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:01.694078 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 00:16:01.694882 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 1448
I20260811 00:16:01.695168 1736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:01.694749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:60921)
0811 00:16:01.694913 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:01.695010 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 00:16:01.695482 1734 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 00:16:01.695726 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:42387
--local_ip_for_outbound_sockets=127.7.94.2
--tserver_master_addrs=127.7.94.62:41023
--webserver_port=35237
--webserver_interface=127.7.94.2
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:16:02.033268 1739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:16:02.033547 1739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:16:02.033782 1739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:16:02.040890 1739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:16:02.041056 1739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:16:02.041297 1739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:16:02.051708 1739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:42387
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=35237
--tserver_master_addrs=127.7.94.62:41023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:16:02.053076 1739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:16:02.054546 1739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:16:02.068383 1751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:02.068325 1755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:02.068325 1759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:16:02.069545 1739 server_base.cc:1034] running on GCE node
I20260811 00:16:02.070176 1739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:16:02.071116 1739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:16:02.072350 1739 hybrid_clock.cc:648] HybridClock initialized: now 1786407362072297 us; error 63 us; skew 500 ppm
I20260811 00:16:02.072674 1739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:16:02 dist-test-slave-8hhm krb5kdc[31983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:16:02.075824 1739 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:16:02.077358 1739 webserver.cc:460] Webserver started at http://127.7.94.2:35237/ using document root <none> and password file <none>
I20260811 00:16:02.078047 1739 fs_manager.cc:359] Metadata directory not provided
I20260811 00:16:02.078194 1739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:16:02.083518 1739 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.004s
I20260811 00:16:02.086300 1769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:02.087543 1739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:16:02.087731 1739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "d6ea2a8ad06d4a208e6408344df581f2"
format_stamp: "Formatted at 2026-08-11 00:15:56 on dist-test-slave-8hhm"
server_key: "79f91ff0f15822ce41dff5d24d57e153"
server_key_iv: "0a5d2b44968b3821c155cd53e6631d6c"
server_key_version: "encryptionkey@0"
I20260811 00:16:02.088693 1739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:16:02.111265 1739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:16:02.115237 1739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:16:02.115537 1739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:16:02.116606 1739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:16:02.117988 1739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:16:02.118139 1739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:02.118268 1739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:16:02.118357 1739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:02.152478 1739 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:42387
I20260811 00:16:02.152535 1965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:42387 every 8 connection(s)
I20260811 00:16:02.154013 1739 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:16:02.156119 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 1739
I20260811 00:16:02.156881 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:38383
--local_ip_for_outbound_sockets=127.7.94.3
--tserver_master_addrs=127.7.94.62:41023
--webserver_port=42085
--webserver_interface=127.7.94.3
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:16:02.159401 1970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:02.157456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.2:38179)
0811 00:16:02.158317 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:02.158512 (+ 195us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":349,"threads_started":1}
W20260811 00:16:02.160032 1968 heartbeater.cc:644] Failed to heartbeat to 127.7.94.62:41023 (0 consecutive failures): Network error: Failed to ping master at 127.7.94.62:41023: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
I20260811 00:16:02.161000 1970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:02.160689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.2:59281)
0811 00:16:02.160833 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:02.160897 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 00:16:02.161976 1970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:02.161652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.2:33857)
0811 00:16:02.161801 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:02.161871 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260811 00:16:02.162228 1968 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 00:16:02.511222 1971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:16:02.511536 1971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:16:02.511633 1971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:16:02.519039 1971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:16:02.519150 1971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:16:02.519258 1971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:16:02.529145 1971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:38383
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=42085
--tserver_master_addrs=127.7.94.62:41023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:16:02.530500 1971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:16:02.532140 1971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:16:02.547320 1981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:02.547288 1985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:02.547355 1982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:16:02.548592 1971 server_base.cc:1034] running on GCE node
I20260811 00:16:02.549278 1971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:16:02.550251 1971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:16:02.551503 1971 hybrid_clock.cc:648] HybridClock initialized: now 1786407362551456 us; error 52 us; skew 500 ppm
I20260811 00:16:02.551855 1971 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:16:02 dist-test-slave-8hhm krb5kdc[31983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:16:02.555258 1971 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:16:02.556805 1971 webserver.cc:460] Webserver started at http://127.7.94.3:42085/ using document root <none> and password file <none>
I20260811 00:16:02.557644 1971 fs_manager.cc:359] Metadata directory not provided
I20260811 00:16:02.557813 1971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:16:02.563263 1971 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 00:16:02.566143 1994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:02.567435 1971 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:16:02.567623 1971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "a432988472e44d3da84446f55752ecf4"
format_stamp: "Formatted at 2026-08-11 00:15:56 on dist-test-slave-8hhm"
server_key: "4057a7c25208bb83a5f9eae1dbcc41a1"
server_key_iv: "14e151daa9e06e5102acf2f6eae1f7eb"
server_key_version: "encryptionkey@0"
I20260811 00:16:02.568640 1971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:16:02.594568 1971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:16:02.599316 1971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:16:02.599624 1971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:16:02.600899 1971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:16:02.602373 1971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:16:02.602525 1971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:02.602654 1971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:16:02.602751 1971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:02.637406 1971 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:38383
I20260811 00:16:02.637497 2197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:38383 every 8 connection(s)
I20260811 00:16:02.638942 1971 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:16:02.644348 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:02.642281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.3:35289)
0811 00:16:02.643232 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:02.643453 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":454,"threads_started":1}
W20260811 00:16:02.644979 2198 heartbeater.cc:644] Failed to heartbeat to 127.7.94.62:41023 (0 consecutive failures): Network error: Failed to ping master at 127.7.94.62:41023: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
I20260811 00:16:02.645802 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:02.645504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.3:43491)
0811 00:16:02.645635 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:02.645699 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 00:16:02.646621 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:02.646326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.3:55723)
0811 00:16:02.646440 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:02.646509 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 00:16:02.646860 2198 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 00:16:02.648751 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 1971
I20260811 00:16:02.696784 2203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:02.695997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:42739)
0811 00:16:02.696435 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:02.696536 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":205,"threads_started":1}
I20260811 00:16:03.163779 2204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:03.162923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.2:39267)
0811 00:16:03.163389 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:03.163477 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":244,"threads_started":1}
I20260811 00:16:03.330174 2205 raft_consensus.cc:482] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:16:03.330538 2205 raft_consensus.cc:504] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:16:03.332140 2205 leader_election.cc:288] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a432988472e44d3da84446f55752ecf4 (127.7.94.3:38383), d6ea2a8ad06d4a208e6408344df581f2 (127.7.94.2:42387)
Aug 11 00:16:03 dist-test-slave-8hhm krb5kdc[31983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407361, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.2@KRBTEST.COM
Aug 11 00:16:03 dist-test-slave-8hhm krb5kdc[31983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407361, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.3@KRBTEST.COM
I20260811 00:16:03.355784 2210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:03.333093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:42387 (local address 127.7.94.1:45291)
0811 00:16:03.333862 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:03.333880 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:16:03.334025 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:16:03.337603 (+ 3578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:16:03.337607 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:16:03.338317 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:16:03.339547 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:03.339576 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:03.341883 (+ 2307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:03.341891 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:03.343323 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:03.343335 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:03.344361 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:03.344367 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:03.344658 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:03.345642 (+ 984us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:16:03.345659 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:16:03.348191 (+ 2532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:16:03.351888 (+ 3697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:16:03.351898 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:16:03.351914 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:16:03.352436 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:16:03.353007 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:16:03.353014 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:16:03.353018 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:16:03.353312 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:16:03.354183 (+ 871us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:16:03.354188 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:16:03.354468 (+ 280us) client_negotiation.cc:769] Sending connection context
0811 00:16:03.354731 (+ 263us) client_negotiation.cc:240] Negotiation successful
0811 00:16:03.355299 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"mutex_wait_us":181,"spinlock_wait_cycles":36608,"thread_start_us":436,"threads_started":1}
I20260811 00:16:03.355842 2207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:03.332667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:50449 (local address 127.7.94.3:38383)
0811 00:16:03.333260 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:03.333278 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:03.333327 (+ 49us) server_negotiation.cc:407] Connection header received
0811 00:16:03.336250 (+ 2923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:03.336262 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:03.336758 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:03.337155 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:03.339799 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:03.342092 (+ 2293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:03.343518 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:03.344127 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:03.344317 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:03.348733 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:03.348741 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:03.348753 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:03.348794 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:03.351764 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:03.352732 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:03.352741 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:03.352748 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:03.352956 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:03.353485 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:03.353490 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:03.353494 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:03.353980 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:03.354197 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:03.355000 (+ 803us) server_negotiation.cc:299] Negotiation successful
0811 00:16:03.355415 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":275,"threads_started":1}
I20260811 00:16:03.355861 2212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:03.333718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:45291 (local address 127.7.94.2:42387)
0811 00:16:03.334551 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:03.334566 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:03.334621 (+ 55us) server_negotiation.cc:407] Connection header received
0811 00:16:03.336802 (+ 2181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:03.336812 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:03.337260 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:03.337635 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:03.339799 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:03.341700 (+ 1901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:03.343471 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:03.344132 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:03.344345 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:03.348491 (+ 4146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:03.348502 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:03.348521 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:03.348573 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:03.351745 (+ 3172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:03.352583 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:03.352592 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:03.352600 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:03.352849 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:03.353432 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:03.353438 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:03.353446 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:03.353986 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:03.354237 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:03.354990 (+ 753us) server_negotiation.cc:299] Negotiation successful
0811 00:16:03.355394 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":460,"threads_started":1}
I20260811 00:16:03.355880 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:03.332409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:38383 (local address 127.7.94.1:50449)
0811 00:16:03.332964 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:03.333002 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 00:16:03.333691 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:16:03.337306 (+ 3615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:16:03.337314 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:16:03.338317 (+ 1003us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:16:03.339548 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:03.339576 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:03.342270 (+ 2694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:03.342274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:03.343339 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:03.343352 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:03.344343 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:03.344350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:03.344661 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:03.345590 (+ 929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:16:03.345614 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:16:03.348479 (+ 2865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:16:03.351928 (+ 3449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:16:03.351932 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:16:03.351934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:16:03.352576 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:16:03.353084 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:16:03.353088 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:16:03.353091 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:16:03.353355 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:16:03.354125 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:16:03.354132 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:16:03.354467 (+ 335us) client_negotiation.cc:769] Sending connection context
0811 00:16:03.354732 (+ 265us) client_negotiation.cc:240] Negotiation successful
0811 00:16:03.355446 (+ 714us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":370,"spinlock_wait_cycles":29952,"thread_start_us":281,"threads_started":1}
W20260811 00:16:03.359586 1533 leader_election.cc:341] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a432988472e44d3da84446f55752ecf4 (127.7.94.3:38383): Not found: Tablet not found: 6942eaafc371478d831097ba1334df66
W20260811 00:16:03.360363 1533 leader_election.cc:341] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6ea2a8ad06d4a208e6408344df581f2 (127.7.94.2:42387): Not found: Tablet not found: 6942eaafc371478d831097ba1334df66
I20260811 00:16:03.360505 1533 leader_election.cc:302] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28832f2904e04b6c86b1b069c2754e46; no voters: a432988472e44d3da84446f55752ecf4, d6ea2a8ad06d4a208e6408344df581f2
I20260811 00:16:03.360917 2205 raft_consensus.cc:2726] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 00:16:03.648345 2213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:03.647425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.3:39965)
0811 00:16:03.647924 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:03.648078 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":263,"threads_started":1}
I20260811 00:16:03.698014 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:03.697643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:59339)
0811 00:16:03.697788 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:03.697879 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 00:16:04.165618 2214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:04.164828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.2:36523)
0811 00:16:04.165332 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:04.165434 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":241,"threads_started":1}
I20260811 00:16:04.650485 2215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:04.649545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.3:60243)
0811 00:16:04.650153 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:04.650265 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":322,"threads_started":1}
I20260811 00:16:04.699573 2219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:04.698856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:52607)
0811 00:16:04.699309 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:04.699391 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":232,"threads_started":1}
I20260811 00:16:05.167063 2221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:05.166411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.2:35417)
0811 00:16:05.166817 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:05.166892 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":210,"threads_started":1}
I20260811 00:16:05.437366 2222 raft_consensus.cc:482] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:16:05.437565 2222 raft_consensus.cc:504] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:16:05.438136 2222 leader_election.cc:288] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a432988472e44d3da84446f55752ecf4 (127.7.94.3:38383), d6ea2a8ad06d4a208e6408344df581f2 (127.7.94.2:42387)
W20260811 00:16:05.439311 1533 leader_election.cc:341] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a432988472e44d3da84446f55752ecf4 (127.7.94.3:38383): Not found: Tablet not found: 6942eaafc371478d831097ba1334df66
W20260811 00:16:05.439601 1533 leader_election.cc:341] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6ea2a8ad06d4a208e6408344df581f2 (127.7.94.2:42387): Not found: Tablet not found: 6942eaafc371478d831097ba1334df66
I20260811 00:16:05.439707 1533 leader_election.cc:302] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28832f2904e04b6c86b1b069c2754e46; no voters: a432988472e44d3da84446f55752ecf4, d6ea2a8ad06d4a208e6408344df581f2
I20260811 00:16:05.439968 2222 raft_consensus.cc:2726] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 00:16:05.652175 2224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:05.651414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.3:49771)
0811 00:16:05.651909 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:05.651996 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":257,"threads_started":1}
I20260811 00:16:05.701422 2226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:05.700551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:39853)
0811 00:16:05.700999 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:05.701145 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":209,"threads_started":1}
I20260811 00:16:06.168771 2227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:06.168084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.2:41533)
0811 00:16:06.168508 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:06.168589 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":214,"threads_started":1}
I20260811 00:16:06.653891 2230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:06.653063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.3:56429)
0811 00:16:06.653619 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:06.653704 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":341,"threads_started":1}
I20260811 00:16:06.703156 2232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:06.702429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:59843)
0811 00:16:06.702879 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:06.702957 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":218,"threads_started":1}
I20260811 00:16:07.170445 2234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:07.169786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.2:47385)
0811 00:16:07.170216 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:07.170291 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":245,"threads_started":1}
I20260811 00:16:07.655771 2236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:07.654841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.3:39325)
0811 00:16:07.655486 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:07.655575 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":261,"threads_started":1}
I20260811 00:16:07.704972 2243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:07.704257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:40623)
0811 00:16:07.704713 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:07.704805 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":249,"threads_started":1}
I20260811 00:16:08.172227 2246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:08.171428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.2:54207)
0811 00:16:08.171925 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:08.172005 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":279,"threads_started":1}
I20260811 00:16:08.484766 2247 raft_consensus.cc:482] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:16:08.484992 2247 raft_consensus.cc:504] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:16:08.485666 2247 leader_election.cc:288] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a432988472e44d3da84446f55752ecf4 (127.7.94.3:38383), d6ea2a8ad06d4a208e6408344df581f2 (127.7.94.2:42387)
W20260811 00:16:08.486680 1533 leader_election.cc:341] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a432988472e44d3da84446f55752ecf4 (127.7.94.3:38383): Not found: Tablet not found: 6942eaafc371478d831097ba1334df66
W20260811 00:16:08.486908 1533 leader_election.cc:341] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6ea2a8ad06d4a208e6408344df581f2 (127.7.94.2:42387): Not found: Tablet not found: 6942eaafc371478d831097ba1334df66
I20260811 00:16:08.487038 1533 leader_election.cc:302] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28832f2904e04b6c86b1b069c2754e46; no voters: a432988472e44d3da84446f55752ecf4, d6ea2a8ad06d4a208e6408344df581f2
I20260811 00:16:08.487311 2247 raft_consensus.cc:2726] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 00:16:08.657612 2249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:08.656780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.3:57971)
0811 00:16:08.657350 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:08.657432 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":267,"threads_started":1}
I20260811 00:16:08.706588 2250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:08.705815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:51749)
0811 00:16:08.706287 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:08.706364 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":233,"threads_started":1}
I20260811 00:16:09.174134 2259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:09.173206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.2:47685)
0811 00:16:09.173850 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:09.173933 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":294,"threads_started":1}
I20260811 00:16:09.659301 2261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:09.658478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.3:37577)
0811 00:16:09.659025 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:09.659105 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":267,"threads_started":1}
I20260811 00:16:09.708225 2262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:09.707523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:34165)
0811 00:16:09.707989 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:09.708072 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":273,"threads_started":1}
I20260811 00:16:10.175762 2264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:10.175018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.2:36827)
0811 00:16:10.175509 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:10.175587 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":277,"threads_started":1}
I20260811 00:16:10.660914 2265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:10.660234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.3:43417)
0811 00:16:10.660648 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:10.660725 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":213,"threads_started":1}
I20260811 00:16:10.709764 2267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:10.709042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:53663)
0811 00:16:10.709518 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:10.709596 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":265,"threads_started":1}
I20260811 00:16:11.177410 2268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:11.176667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.2:54397)
0811 00:16:11.177095 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:11.177187 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":211,"threads_started":1}
I20260811 00:16:11.662853 2269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:11.661972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.3:41791)
0811 00:16:11.662552 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:11.662649 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":272,"threads_started":1}
I20260811 00:16:11.711856 2270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:11.710830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:55685)
0811 00:16:11.711458 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:11.711580 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":304,"threads_started":1}
I20260811 00:16:12.179153 2271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:12.178349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.2:52273)
0811 00:16:12.178890 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:12.178975 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":318,"threads_started":1}
I20260811 00:16:12.664600 2273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:12.663878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.3:51467)
0811 00:16:12.664323 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:12.664401 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":232,"threads_started":1}
I20260811 00:16:12.713505 2274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:12.712715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:35885)
0811 00:16:12.713163 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:12.713303 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":239,"threads_started":1}
I20260811 00:16:12.891886 2275 raft_consensus.cc:482] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:16:12.892114 2275 raft_consensus.cc:504] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:16:12.892851 2275 leader_election.cc:288] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a432988472e44d3da84446f55752ecf4 (127.7.94.3:38383), d6ea2a8ad06d4a208e6408344df581f2 (127.7.94.2:42387)
W20260811 00:16:12.893832 1533 leader_election.cc:341] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a432988472e44d3da84446f55752ecf4 (127.7.94.3:38383): Not found: Tablet not found: 6942eaafc371478d831097ba1334df66
W20260811 00:16:12.894028 1533 leader_election.cc:341] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6ea2a8ad06d4a208e6408344df581f2 (127.7.94.2:42387): Not found: Tablet not found: 6942eaafc371478d831097ba1334df66
I20260811 00:16:12.894110 1533 leader_election.cc:302] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28832f2904e04b6c86b1b069c2754e46; no voters: a432988472e44d3da84446f55752ecf4, d6ea2a8ad06d4a208e6408344df581f2
I20260811 00:16:12.894333 2275 raft_consensus.cc:2726] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 00:16:13.180927 2277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:13.180201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.2:38289)
0811 00:16:13.180680 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:13.180760 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":238,"threads_started":1}
I20260811 00:16:13.666393 2278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:13.665609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.3:40241)
0811 00:16:13.666146 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:13.666234 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":265,"threads_started":1}
I20260811 00:16:13.715461 2280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:13.714407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:42129)
0811 00:16:13.715078 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:13.715239 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":422,"threads_started":1}
I20260811 00:16:14.182751 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:14.182018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.2:43039)
0811 00:16:14.182483 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:14.182566 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":238,"threads_started":1}
I20260811 00:16:14.668107 2283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:14.667361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.3:53709)
0811 00:16:14.667841 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:14.667923 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":244,"threads_started":1}
I20260811 00:16:14.717149 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:14.716398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:53625)
0811 00:16:14.716868 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:14.716962 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":257,"threads_started":1}
I20260811 00:16:15.184428 2286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:15.183691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.2:37761)
0811 00:16:15.184181 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:15.184264 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":242,"threads_started":1}
I20260811 00:16:15.670007 2289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:15.669002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.3:38101)
0811 00:16:15.669573 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:15.669728 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":309,"threads_started":1}
I20260811 00:16:15.718889 2290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:15.718187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:46283)
0811 00:16:15.718605 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:15.718697 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":222,"threads_started":1}
I20260811 00:16:16.186192 2291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:16.185438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.2:55049)
0811 00:16:16.185905 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:16.185986 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":243,"threads_started":1}
I20260811 00:16:16.671860 2292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:16.671022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.3:49575)
0811 00:16:16.671551 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:16.671635 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":238,"threads_started":1}
I20260811 00:16:16.720760 2294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:16.720094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:49475)
0811 00:16:16.720526 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:16.720594 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":200,"threads_started":1}
I20260811 00:16:17.011842 2297 raft_consensus.cc:482] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:16:17.012040 2297 raft_consensus.cc:504] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a432988472e44d3da84446f55752ecf4" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 38383 } } peers { permanent_uuid: "d6ea2a8ad06d4a208e6408344df581f2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 42387 } } peers { permanent_uuid: "28832f2904e04b6c86b1b069c2754e46" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 41895 } }
I20260811 00:16:17.012626 2297 leader_election.cc:288] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a432988472e44d3da84446f55752ecf4 (127.7.94.3:38383), d6ea2a8ad06d4a208e6408344df581f2 (127.7.94.2:42387)
W20260811 00:16:17.013696 1533 leader_election.cc:341] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a432988472e44d3da84446f55752ecf4 (127.7.94.3:38383): Not found: Tablet not found: 6942eaafc371478d831097ba1334df66
W20260811 00:16:17.013885 1533 leader_election.cc:341] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d6ea2a8ad06d4a208e6408344df581f2 (127.7.94.2:42387): Not found: Tablet not found: 6942eaafc371478d831097ba1334df66
I20260811 00:16:17.013970 1533 leader_election.cc:302] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28832f2904e04b6c86b1b069c2754e46; no voters: a432988472e44d3da84446f55752ecf4, d6ea2a8ad06d4a208e6408344df581f2
I20260811 00:16:17.014181 2297 raft_consensus.cc:2726] T 6942eaafc371478d831097ba1334df66 P 28832f2904e04b6c86b1b069c2754e46 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 00:16:17.188043 2300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:17.187332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.2:33941)
0811 00:16:17.187795 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:17.187887 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":240,"threads_started":1}
I20260811 00:16:17.656383 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 1448
I20260811 00:16:17.670876 2301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:17.657679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33696 (local address 127.7.94.1:41895)
0811 00:16:17.658171 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:17.658179 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:17.658205 (+ 26us) server_negotiation.cc:407] Connection header received
0811 00:16:17.658269 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:17.658276 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:17.658495 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:17.658625 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:17.660049 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:17.660974 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:17.662195 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:17.662713 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:17.662807 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:17.665430 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:17.665440 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:17.665454 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:17.665489 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:17.668313 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:17.668904 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:17.668910 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:17.668912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:17.669053 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:17.669485 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:17.669488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:17.669490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:17.669835 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:17.670064 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:17.670295 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 00:16:17.670529 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":242,"threads_started":1}
I20260811 00:16:17.672384 1725 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:16:17.674217 2303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:17.673131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.3:41509)
0811 00:16:17.673803 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:17.673952 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":290,"threads_started":1}
I20260811 00:16:17.762238 2306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:17.761112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41023 (local address 127.7.94.1:40931)
0811 00:16:17.761710 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:17.761868 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:41023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":349,"threads_started":1}
I20260811 00:16:17.762491 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 1448
I20260811 00:16:17.790215 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 1739
I20260811 00:16:17.803720 2307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:17.791011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59146 (local address 127.7.94.2:42387)
0811 00:16:17.791512 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:17.791517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:17.791534 (+ 17us) server_negotiation.cc:407] Connection header received
0811 00:16:17.791595 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:17.791611 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:17.791672 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:17.791796 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:17.793445 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:17.794224 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:17.795396 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:17.795814 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:17.795908 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:17.798699 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:17.798704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:17.798707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:17.798738 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:17.801442 (+ 2704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:17.802080 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:17.802083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:17.802086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:17.802221 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:17.802531 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:17.802534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:17.802536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:17.802812 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:17.803025 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:17.803234 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 00:16:17.803411 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":294,"threads_started":1}
I20260811 00:16:17.804472 1960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:16:17.888609 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 1739
I20260811 00:16:17.919250 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 1971
I20260811 00:16:17.934178 2312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:17.920139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45292 (local address 127.7.94.3:38383)
0811 00:16:17.920853 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:17.920858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:17.920876 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:16:17.920935 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:17.920939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:17.921122 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:17.921296 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:17.922632 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:17.923411 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:17.924604 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:17.925003 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:17.925093 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:17.927744 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:17.927749 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:17.927752 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:17.927784 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:17.931194 (+ 3410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:17.932029 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:17.932035 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:17.932038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:17.932184 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:17.932608 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:17.932613 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:17.932616 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:17.932909 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:17.933244 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:17.933542 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 00:16:17.933779 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":422,"threads_started":1}
I20260811 00:16:17.934902 2190 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:16:18.019114 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 1971
2026-08-11T00:16:18Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24963 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:16:18 dist-test-slave-8hhm krb5kdc[2318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:16:18 dist-test-slave-8hhm krb5kdc[2318](info): set up 2 sockets
Aug 11 00:16:18 dist-test-slave-8hhm krb5kdc[2318](info): commencing operation
krb5kdc: starting...
W20260811 00:16:19.383565 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:16:19 dist-test-slave-8hhm krb5kdc[2318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:16:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:16:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:16:19.691044 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:40801
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:40801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:16:20.045660 2340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:16:20.045949 2340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:16:20.046041 2340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:16:20.053138 2340 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 00:16:20.053263 2340 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 00:16:20.053303 2340 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 00:16:20.053355 2340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:16:20.053406 2340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:16:20.053471 2340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:16:20.053516 2340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:16:20.053561 2340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:16:20.063956 2340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:40801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:40801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:16:20.065326 2340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:16:20.066695 2340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:16:20.079159 2350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:20.079233 2348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:20.079196 2352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:16:20.079784 2340 server_base.cc:1034] running on GCE node
I20260811 00:16:20.080765 2340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:16:20.082321 2340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:16:20.083573 2340 hybrid_clock.cc:648] HybridClock initialized: now 1786407380083489 us; error 59 us; skew 500 ppm
I20260811 00:16:20.084004 2340 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:16:20 dist-test-slave-8hhm krb5kdc[2318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407380, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:16:20.087473 2340 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:16:20.089005 2340 webserver.cc:460] Webserver started at http://127.7.94.62:46187/ using document root <none> and password file <none>
I20260811 00:16:20.089835 2340 fs_manager.cc:359] Metadata directory not provided
I20260811 00:16:20.089968 2340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:16:20.090335 2340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:16:20.093547 2340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "4d5f62bf2ddd4264b716ec18b3554d22"
format_stamp: "Formatted at 2026-08-11 00:16:20 on dist-test-slave-8hhm"
server_key: "9c3059d80090978456b5302472bd99c1"
server_key_iv: "6c5f2409687018dc568f8e554105130a"
server_key_version: "encryptionkey@0"
I20260811 00:16:20.094457 2340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "4d5f62bf2ddd4264b716ec18b3554d22"
format_stamp: "Formatted at 2026-08-11 00:16:20 on dist-test-slave-8hhm"
server_key: "9c3059d80090978456b5302472bd99c1"
server_key_iv: "6c5f2409687018dc568f8e554105130a"
server_key_version: "encryptionkey@0"
I20260811 00:16:20.099888 2340 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:16:20.103381 2360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:20.104708 2340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 00:16:20.104910 2340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "4d5f62bf2ddd4264b716ec18b3554d22"
format_stamp: "Formatted at 2026-08-11 00:16:20 on dist-test-slave-8hhm"
server_key: "9c3059d80090978456b5302472bd99c1"
server_key_iv: "6c5f2409687018dc568f8e554105130a"
server_key_version: "encryptionkey@0"
I20260811 00:16:20.105170 2340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:16:20.141608 2340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:16:20.146837 2340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:16:20.147166 2340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:16:20.167752 2340 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:40801
I20260811 00:16:20.167783 2446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:40801 every 8 connection(s)
I20260811 00:16:20.169420 2340 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:16:20.173399 2448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:16:20.173425 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 2340
I20260811 00:16:20.173645 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:16:20.174335 7544 external_mini_cluster.cc:1428] Setting key b61a73f22ababdae7c9f1a0e5897b3eb
I20260811 00:16:20.188143 2448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22: Bootstrap starting.
Aug 11 00:16:20 dist-test-slave-8hhm krb5kdc[2318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:16:20.191488 2448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22: Neither blocks nor log segments found. Creating new log.
I20260811 00:16:20.192802 2448 log.cc:824] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22: Log is configured to *not* fsync() on all Append() calls
I20260811 00:16:20.196406 2448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22: No bootstrap required, opened a new log
I20260811 00:16:20.200737 2454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:20.176495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34774 (local address 127.7.94.62:40801)
0811 00:16:20.177398 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:20.177413 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:20.177470 (+ 57us) server_negotiation.cc:407] Connection header received
0811 00:16:20.180233 (+ 2763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:20.180265 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:20.180822 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:20.181324 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:20.182832 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:20.184814 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:20.186305 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:20.187348 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:20.187524 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:20.192100 (+ 4576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:20.192111 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:20.192128 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:20.192177 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:20.196218 (+ 4041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:20.197018 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:20.197026 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:20.197034 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:20.197269 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:20.197669 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:20.197674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:20.197677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:20.198379 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:20.198709 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:20.198994 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 00:16:20.199427 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":361,"threads_started":1}
I20260811 00:16:20.205878 2448 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40801 } }
I20260811 00:16:20.206208 2448 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:16:20.206274 2448 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d5f62bf2ddd4264b716ec18b3554d22, State: Initialized, Role: FOLLOWER
I20260811 00:16:20.206827 2448 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40801 } }
I20260811 00:16:20.207075 2448 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:16:20.207221 2448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:16:20.207413 2448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:16:20.209684 2448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40801 } }
I20260811 00:16:20.210170 2448 leader_election.cc:302] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d5f62bf2ddd4264b716ec18b3554d22; no voters:
I20260811 00:16:20.210783 2448 leader_election.cc:288] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:16:20.210939 2456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:16:20.211689 2456 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 1 LEADER]: Becoming Leader. State: Replica: 4d5f62bf2ddd4264b716ec18b3554d22, State: Running, Role: LEADER
I20260811 00:16:20.212275 2456 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40801 } }
I20260811 00:16:20.213179 2448 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:16:20.217533 2458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d5f62bf2ddd4264b716ec18b3554d22. Latest consensus state: current_term: 1 leader_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40801 } } }
I20260811 00:16:20.217888 2458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [sys.catalog]: This master's current role is: LEADER
I20260811 00:16:20.218365 2460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40801 } } }
I20260811 00:16:20.218712 2460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [sys.catalog]: This master's current role is: LEADER
I20260811 00:16:20.222658 2463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:16:20.228075 2463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:16:20.237128 2463 catalog_manager.cc:1347] Generated new cluster ID: 5eaf0bddeab44da78a5fcacb99d6b83f
I20260811 00:16:20.237263 2463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:16:20.270305 2463 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:16:20.271727 2463 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:16:20.278277 2463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22: Generated new TSK 0
I20260811 00:16:20.279085 2463 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
I20260811 00:16:20.353627 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:40801
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:16:20.704563 2491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:16:20.704922 2491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:16:20.705112 2491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:16:20.712854 2491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:16:20.713027 2491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:16:20.713321 2491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:16:20.723775 2491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:40801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:16:20.725137 2491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:16:20.726681 2491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:16:20.740830 2499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:20.740901 2501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:20.740868 2504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:16:20.741714 2491 server_base.cc:1034] running on GCE node
I20260811 00:16:20.742385 2491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:16:20.743359 2491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:16:20.744553 2491 hybrid_clock.cc:648] HybridClock initialized: now 1786407380744515 us; error 38 us; skew 500 ppm
I20260811 00:16:20.744875 2491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:16:20 dist-test-slave-8hhm krb5kdc[2318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407380, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:16:20.751958 2491 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:16:20.753527 2491 webserver.cc:460] Webserver started at http://127.7.94.1:35333/ using document root <none> and password file <none>
I20260811 00:16:20.754228 2491 fs_manager.cc:359] Metadata directory not provided
I20260811 00:16:20.754376 2491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:16:20.754703 2491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:16:20.757740 2491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "64f39c6323ef42a8842b3d870b90ce2e"
format_stamp: "Formatted at 2026-08-11 00:16:20 on dist-test-slave-8hhm"
server_key: "b21b4bf7975842607252b11e3eeb3642"
server_key_iv: "8c91d7630c72243ae736ed371d510efd"
server_key_version: "encryptionkey@0"
I20260811 00:16:20.758494 2491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "64f39c6323ef42a8842b3d870b90ce2e"
format_stamp: "Formatted at 2026-08-11 00:16:20 on dist-test-slave-8hhm"
server_key: "b21b4bf7975842607252b11e3eeb3642"
server_key_iv: "8c91d7630c72243ae736ed371d510efd"
server_key_version: "encryptionkey@0"
I20260811 00:16:20.763665 2491 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:16:20.767094 2516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:20.768374 2491 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 00:16:20.768553 2491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "64f39c6323ef42a8842b3d870b90ce2e"
format_stamp: "Formatted at 2026-08-11 00:16:20 on dist-test-slave-8hhm"
server_key: "b21b4bf7975842607252b11e3eeb3642"
server_key_iv: "8c91d7630c72243ae736ed371d510efd"
server_key_version: "encryptionkey@0"
I20260811 00:16:20.768779 2491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:16:20.785775 2491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:16:20.792510 2491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:16:20.792815 2491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:16:20.793911 2491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:16:20.795295 2491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:16:20.795401 2491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:20.795558 2491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:16:20.795639 2491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:20.830122 2491 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:42897
I20260811 00:16:20.830163 2718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:42897 every 8 connection(s)
I20260811 00:16:20.831461 2491 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:16:20.840387 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 2491
I20260811 00:16:20.840637 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:16:20.841362 7544 external_mini_cluster.cc:1428] Setting key 983161ddbd72684a58789b3414c11c68
Aug 11 00:16:20 dist-test-slave-8hhm krb5kdc[2318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407380, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:16:20.853168 2724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:20.834113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:60815 (local address 127.7.94.62:40801)
0811 00:16:20.834564 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:20.834571 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:20.835677 (+ 1106us) server_negotiation.cc:407] Connection header received
0811 00:16:20.837513 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:20.837518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:20.837585 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:20.837682 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:20.839238 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:20.839996 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:20.841795 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:20.842314 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:20.842440 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:20.846262 (+ 3822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:20.846271 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:20.846278 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:20.846322 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:20.849902 (+ 3580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:20.850614 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:20.850621 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:20.850627 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:20.850782 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:20.851206 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:20.851213 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:20.851219 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:20.851580 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:20.851764 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:20.852457 (+ 693us) server_negotiation.cc:299] Negotiation successful
0811 00:16:20.852703 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":208,"threads_started":1}
I20260811 00:16:20.853799 2725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:20.834796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.1:60815)
0811 00:16:20.835506 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:20.835544 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 00:16:20.837199 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:16:20.837899 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:16:20.837906 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:16:20.838435 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:16:20.839002 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:20.839021 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:20.840155 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:20.840158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:20.841644 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:20.841666 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:20.842557 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:20.842563 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:20.842854 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:20.843661 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:16:20.843689 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:16:20.846046 (+ 2357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:16:20.850036 (+ 3990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:16:20.850042 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:16:20.850056 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:16:20.850482 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:16:20.850910 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:16:20.850913 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:16:20.850916 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:16:20.851090 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:16:20.851711 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:16:20.851717 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:16:20.852050 (+ 333us) client_negotiation.cc:769] Sending connection context
0811 00:16:20.852272 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 00:16:20.852680 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":296,"threads_started":1}
I20260811 00:16:20.855283 2719 heartbeater.cc:347] Connected to a master server at 127.7.94.62:40801
I20260811 00:16:20.855594 2719 heartbeater.cc:464] Registering TS with master...
I20260811 00:16:20.856328 2719 heartbeater.cc:511] Master 127.7.94.62:40801 requested a full tablet report, sending...
I20260811 00:16:20.858631 2385 ts_manager.cc:194] Registered new tserver with Master: 64f39c6323ef42a8842b3d870b90ce2e (127.7.94.1:42897)
I20260811 00:16:20.859995 2385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:60815
WARNING: no policy specified for kudu/127.7.94.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
I20260811 00:16:20.910573 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:40801
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:16:21.264031 2729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:16:21.264400 2729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:16:21.264571 2729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:16:21.271991 2729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:16:21.272169 2729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:16:21.272378 2729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:16:21.282516 2729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:40801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:16:21.283838 2729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:16:21.285187 2729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:16:21.298616 2734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:21.298552 2735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:21.299081 2739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:16:21.300108 2729 server_base.cc:1034] running on GCE node
I20260811 00:16:21.300748 2729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:16:21.301754 2729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:16:21.302956 2729 hybrid_clock.cc:648] HybridClock initialized: now 1786407381302911 us; error 32 us; skew 500 ppm
I20260811 00:16:21.303287 2729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:16:21 dist-test-slave-8hhm krb5kdc[2318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407381, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:16:21.306627 2729 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:16:21.308105 2729 webserver.cc:460] Webserver started at http://127.7.94.2:43887/ using document root <none> and password file <none>
I20260811 00:16:21.308815 2729 fs_manager.cc:359] Metadata directory not provided
I20260811 00:16:21.308924 2729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:16:21.309206 2729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:16:21.312230 2729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "b104e0a1d2034c489e89a15447205e59"
format_stamp: "Formatted at 2026-08-11 00:16:21 on dist-test-slave-8hhm"
server_key: "51f96b77b91813a53e5180888c06ebb8"
server_key_iv: "c3d653e05b04f6e31eb669ff31578dec"
server_key_version: "encryptionkey@0"
I20260811 00:16:21.313007 2729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "b104e0a1d2034c489e89a15447205e59"
format_stamp: "Formatted at 2026-08-11 00:16:21 on dist-test-slave-8hhm"
server_key: "51f96b77b91813a53e5180888c06ebb8"
server_key_iv: "c3d653e05b04f6e31eb669ff31578dec"
server_key_version: "encryptionkey@0"
I20260811 00:16:21.318526 2729 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 00:16:21.321982 2755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:21.323370 2729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:16:21.323542 2729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "b104e0a1d2034c489e89a15447205e59"
format_stamp: "Formatted at 2026-08-11 00:16:21 on dist-test-slave-8hhm"
server_key: "51f96b77b91813a53e5180888c06ebb8"
server_key_iv: "c3d653e05b04f6e31eb669ff31578dec"
server_key_version: "encryptionkey@0"
I20260811 00:16:21.323741 2729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:16:21.346657 2729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:16:21.351238 2729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:16:21.351542 2729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:16:21.352552 2729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:16:21.353933 2729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:16:21.354038 2729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:21.354188 2729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:16:21.354262 2729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:21.388283 2729 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:35975
I20260811 00:16:21.388331 2940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:35975 every 8 connection(s)
I20260811 00:16:21.389719 2729 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:16:21.390866 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 2729
I20260811 00:16:21.391069 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:16:21.391670 7544 external_mini_cluster.cc:1428] Setting key 7bd3415d9332398f147baaa2a62cc192
Aug 11 00:16:21 dist-test-slave-8hhm krb5kdc[2318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407381, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:16:21.414390 2943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:21.392704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:54601 (local address 127.7.94.62:40801)
0811 00:16:21.393199 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:21.393205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:21.394463 (+ 1258us) server_negotiation.cc:407] Connection header received
0811 00:16:21.397454 (+ 2991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:21.397458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:21.397528 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:21.397622 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:21.399833 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:21.400548 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:21.403190 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:21.403622 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:21.403729 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:21.408046 (+ 4317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:21.408050 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:21.408053 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:21.408105 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:21.411175 (+ 3070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:21.411952 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:21.411957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:21.411961 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:21.412098 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:21.412541 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:21.412546 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:21.412549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:21.412813 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:21.412969 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:21.413750 (+ 781us) server_negotiation.cc:299] Negotiation successful
0811 00:16:21.413999 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":251,"threads_started":1}
I20260811 00:16:21.415412 2945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:21.393401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.2:54601)
0811 00:16:21.394265 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:21.394329 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 00:16:21.397119 (+ 2790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:16:21.397922 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:16:21.397936 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:16:21.398787 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:16:21.399638 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:21.399669 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:21.400710 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:21.400719 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:21.403023 (+ 2304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:21.403041 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:21.403865 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:21.403874 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:21.404214 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:21.404985 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:16:21.405020 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:16:21.407810 (+ 2790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:16:21.411315 (+ 3505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:16:21.411324 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:16:21.411341 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:16:21.411820 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:16:21.412221 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:16:21.412227 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:16:21.412230 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:16:21.412420 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:16:21.412944 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:16:21.412951 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:16:21.413362 (+ 411us) client_negotiation.cc:769] Sending connection context
0811 00:16:21.413630 (+ 268us) client_negotiation.cc:240] Negotiation successful
0811 00:16:21.414186 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":337,"threads_started":1}
I20260811 00:16:21.416960 2941 heartbeater.cc:347] Connected to a master server at 127.7.94.62:40801
I20260811 00:16:21.417337 2941 heartbeater.cc:464] Registering TS with master...
I20260811 00:16:21.418083 2941 heartbeater.cc:511] Master 127.7.94.62:40801 requested a full tablet report, sending...
I20260811 00:16:21.419646 2385 ts_manager.cc:194] Registered new tserver with Master: b104e0a1d2034c489e89a15447205e59 (127.7.94.2:35975)
I20260811 00:16:21.420480 2385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:54601
WARNING: no policy specified for kudu/127.7.94.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
I20260811 00:16:21.462997 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:40801
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:16:21.812777 2948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:16:21.813169 2948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:16:21.813371 2948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:16:21.820643 2948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:16:21.820787 2948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:16:21.820971 2948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:16:21.831218 2948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:40801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:16:21.832491 2948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:16:21.833989 2948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:16:21.847241 2965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:21.847187 2961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:21.847190 2959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:16:21.848184 2948 server_base.cc:1034] running on GCE node
I20260811 00:16:21.848798 2948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:16:21.849793 2948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:16:21.851037 2948 hybrid_clock.cc:648] HybridClock initialized: now 1786407381850995 us; error 51 us; skew 500 ppm
I20260811 00:16:21.851361 2948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:16:21 dist-test-slave-8hhm krb5kdc[2318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407381, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:16:21.854712 2948 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:16:21.856115 2948 webserver.cc:460] Webserver started at http://127.7.94.3:45553/ using document root <none> and password file <none>
I20260811 00:16:21.856837 2948 fs_manager.cc:359] Metadata directory not provided
I20260811 00:16:21.856989 2948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:16:21.857381 2948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:16:21.860241 2948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "2b77d2e865944dc0b522cf39c4aeeb92"
format_stamp: "Formatted at 2026-08-11 00:16:21 on dist-test-slave-8hhm"
server_key: "b9f16ee39b6288354622e48e5f3fb461"
server_key_iv: "f20c3d7ec890349495f7699d9f6fdfbd"
server_key_version: "encryptionkey@0"
I20260811 00:16:21.861006 2948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "2b77d2e865944dc0b522cf39c4aeeb92"
format_stamp: "Formatted at 2026-08-11 00:16:21 on dist-test-slave-8hhm"
server_key: "b9f16ee39b6288354622e48e5f3fb461"
server_key_iv: "f20c3d7ec890349495f7699d9f6fdfbd"
server_key_version: "encryptionkey@0"
I20260811 00:16:21.862854 2719 heartbeater.cc:503] Master 127.7.94.62:40801 was elected leader, sending a full tablet report...
I20260811 00:16:21.866474 2948 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 00:16:21.869957 2976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:21.871327 2948 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:16:21.871491 2948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "2b77d2e865944dc0b522cf39c4aeeb92"
format_stamp: "Formatted at 2026-08-11 00:16:21 on dist-test-slave-8hhm"
server_key: "b9f16ee39b6288354622e48e5f3fb461"
server_key_iv: "f20c3d7ec890349495f7699d9f6fdfbd"
server_key_version: "encryptionkey@0"
I20260811 00:16:21.871716 2948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:16:21.897632 2948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:16:21.901695 2948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:16:21.902001 2948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:16:21.903000 2948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:16:21.904361 2948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:16:21.904438 2948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:21.904613 2948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:16:21.904697 2948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:21.938858 2948 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:42797
I20260811 00:16:21.938903 3170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:42797 every 8 connection(s)
I20260811 00:16:21.940253 2948 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:16:21.945640 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 2948
I20260811 00:16:21.945861 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:16:21.946439 7544 external_mini_cluster.cc:1428] Setting key 93db44c9b148a21f6c08cea475159e4b
Aug 11 00:16:21 dist-test-slave-8hhm krb5kdc[2318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407381, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:16:21.960397 3175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:21.942932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:48881 (local address 127.7.94.62:40801)
0811 00:16:21.943430 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:21.943435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:21.944426 (+ 991us) server_negotiation.cc:407] Connection header received
0811 00:16:21.946781 (+ 2355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:21.946786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:21.946855 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:21.946966 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:21.948821 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:21.949827 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:21.951400 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:21.951791 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:21.951886 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:21.955165 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:21.955170 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:21.955173 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:21.955205 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:21.957703 (+ 2498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:21.958338 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:21.958341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:21.958344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:21.958456 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:21.958834 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:21.958837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:21.958839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:21.959097 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:21.959207 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:21.959798 (+ 591us) server_negotiation.cc:299] Negotiation successful
0811 00:16:21.960016 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":256,"threads_started":1}
I20260811 00:16:21.961381 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:21.943534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.3:48881)
0811 00:16:21.944260 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:21.944303 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 00:16:21.946478 (+ 2175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:16:21.947223 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:16:21.947231 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:16:21.947900 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:16:21.948634 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:21.948653 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:21.949958 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:21.949962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:21.951269 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:21.951286 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:21.951971 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:21.951975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:21.952168 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:21.952783 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:16:21.952804 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:16:21.954973 (+ 2169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:16:21.957824 (+ 2851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:16:21.957830 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:16:21.957841 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:16:21.958237 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:16:21.958549 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:16:21.958552 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:16:21.958555 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:16:21.958735 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:16:21.959201 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:16:21.959206 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:16:21.959488 (+ 282us) client_negotiation.cc:769] Sending connection context
0811 00:16:21.959709 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 00:16:21.960062 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":279,"threads_started":1}
I20260811 00:16:21.962764 3172 heartbeater.cc:347] Connected to a master server at 127.7.94.62:40801
I20260811 00:16:21.963027 3172 heartbeater.cc:464] Registering TS with master...
I20260811 00:16:21.963685 3172 heartbeater.cc:511] Master 127.7.94.62:40801 requested a full tablet report, sending...
I20260811 00:16:21.964989 2385 ts_manager.cc:194] Registered new tserver with Master: 2b77d2e865944dc0b522cf39c4aeeb92 (127.7.94.3:42797)
I20260811 00:16:21.965816 2385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:48881
I20260811 00:16:21.974290 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:16:21.991405 3175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:21.979011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53754 (local address 127.7.94.62:40801)
0811 00:16:21.979178 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:21.979182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:21.979633 (+ 451us) server_negotiation.cc:407] Connection header received
0811 00:16:21.979765 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:21.979768 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:21.979828 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:21.979909 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:21.981397 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:21.982160 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:21.983478 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:21.983866 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:21.983958 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:21.986300 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:21.986305 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:21.986309 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:21.986339 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:21.989053 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:21.989786 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:21.989789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:21.989791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:21.989909 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:21.990240 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:21.990243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:21.990245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:21.990547 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:21.990747 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:21.991019 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 00:16:21.991186 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 00:16:21.995414 2385 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:16:22.002261 2385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:16:22.038157 3196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:22.014562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55104 (local address 127.7.94.3:42797)
0811 00:16:22.024912 (+ 10350us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:22.024921 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:22.030117 (+ 5196us) server_negotiation.cc:407] Connection header received
0811 00:16:22.030230 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:22.030237 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:22.030459 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:22.031444 (+ 985us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:16:22.031792 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.033473 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.035613 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.036937 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.037082 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:22.037352 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:22.037637 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 00:16:22.037805 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10159,"thread_start_us":9982,"threads_started":1}
I20260811 00:16:22.038204 3194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:22.014198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:42797 (local address 127.0.0.1:55104)
0811 00:16:22.025281 (+ 11083us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:22.025317 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 00:16:22.025510 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:16:22.030685 (+ 5175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:16:22.030689 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:16:22.030718 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:16:22.031211 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:22.031224 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.033613 (+ 2389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.033619 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:22.035434 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:22.035446 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.037149 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.037156 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:22.037389 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:22.037445 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 00:16:22.037584 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 00:16:22.037771 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10823,"mutex_wait_us":4,"thread_start_us":10746,"threads_started":1}
I20260811 00:16:22.039256 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:22.025776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:35975 (local address 127.0.0.1:37382)
0811 00:16:22.029235 (+ 3459us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:22.029257 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:16:22.029388 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:16:22.031862 (+ 2474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:16:22.031869 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:16:22.031897 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:16:22.032300 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:22.032312 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.034267 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.034272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:22.036457 (+ 2185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:22.036472 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.038482 (+ 2010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.038488 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:22.038671 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:22.038703 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 00:16:22.038799 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 00:16:22.038920 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3289,"thread_start_us":2296,"threads_started":1}
I20260811 00:16:22.039348 3200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:22.026059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37382 (local address 127.7.94.2:35975)
0811 00:16:22.031162 (+ 5103us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:22.031172 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:22.031198 (+ 26us) server_negotiation.cc:407] Connection header received
0811 00:16:22.031291 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:22.031298 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:22.031529 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:22.031746 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:16:22.032507 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.034108 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.036632 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.038058 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.038166 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:22.038340 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:22.038884 (+ 544us) server_negotiation.cc:299] Negotiation successful
0811 00:16:22.039050 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4910,"thread_start_us":4763,"threads_started":1}
I20260811 00:16:22.039682 3199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:22.014445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:42897 (local address 127.0.0.1:34570)
0811 00:16:22.028318 (+ 13873us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:22.028340 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:16:22.028499 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:16:22.030523 (+ 2024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:16:22.030536 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:16:22.030588 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:16:22.031224 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:22.031236 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.033515 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.033523 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:22.036157 (+ 2634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:22.036172 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.038689 (+ 2517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.038695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:22.038871 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:22.038899 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 00:16:22.038963 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 00:16:22.039082 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13522,"mutex_wait_us":103,"thread_start_us":14436,"threads_started":1}
I20260811 00:16:22.040627 3197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:22.014851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34570 (local address 127.7.94.1:42897)
0811 00:16:22.029815 (+ 14964us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:22.029823 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:22.029852 (+ 29us) server_negotiation.cc:407] Connection header received
0811 00:16:22.029976 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:22.029984 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:22.030275 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:22.030478 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:16:22.031442 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.033357 (+ 1915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.036338 (+ 2981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.037904 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.038039 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:22.040079 (+ 2040us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:22.040155 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 00:16:22.040312 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14775,"thread_start_us":14784,"threads_started":1}
I20260811 00:16:22.042835 2830 tablet_service.cc:1467] Processing CreateTablet for tablet e8efc7418abc4a18a7ef3793728afebe (DEFAULT_TABLE table=test-table [id=e2096dff9f904239bd97d9946e5f9133]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:16:22.042868 3064 tablet_service.cc:1467] Processing CreateTablet for tablet e8efc7418abc4a18a7ef3793728afebe (DEFAULT_TABLE table=test-table [id=e2096dff9f904239bd97d9946e5f9133]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:16:22.044119 2830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8efc7418abc4a18a7ef3793728afebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:16:22.044322 3064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8efc7418abc4a18a7ef3793728afebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:16:22.045083 2597 tablet_service.cc:1467] Processing CreateTablet for tablet e8efc7418abc4a18a7ef3793728afebe (DEFAULT_TABLE table=test-table [id=e2096dff9f904239bd97d9946e5f9133]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:16:22.046504 2597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8efc7418abc4a18a7ef3793728afebe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:16:22.055774 3204 tablet_bootstrap.cc:492] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59: Bootstrap starting.
I20260811 00:16:22.059489 3204 tablet_bootstrap.cc:654] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59: Neither blocks nor log segments found. Creating new log.
I20260811 00:16:22.060580 3204 log.cc:824] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59: Log is configured to *not* fsync() on all Append() calls
I20260811 00:16:22.063581 3204 tablet_bootstrap.cc:492] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59: No bootstrap required, opened a new log
I20260811 00:16:22.063818 3207 tablet_bootstrap.cc:492] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92: Bootstrap starting.
I20260811 00:16:22.063884 3204 ts_tablet_manager.cc:1397] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.008s
I20260811 00:16:22.065907 3209 tablet_bootstrap.cc:492] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e: Bootstrap starting.
I20260811 00:16:22.067715 3207 tablet_bootstrap.cc:654] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92: Neither blocks nor log segments found. Creating new log.
I20260811 00:16:22.069016 3207 log.cc:824] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92: Log is configured to *not* fsync() on all Append() calls
I20260811 00:16:22.069084 3209 tablet_bootstrap.cc:654] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e: Neither blocks nor log segments found. Creating new log.
I20260811 00:16:22.070408 3209 log.cc:824] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e: Log is configured to *not* fsync() on all Append() calls
I20260811 00:16:22.072458 3207 tablet_bootstrap.cc:492] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92: No bootstrap required, opened a new log
I20260811 00:16:22.072762 3207 ts_tablet_manager.cc:1397] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260811 00:16:22.073144 3209 tablet_bootstrap.cc:492] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e: No bootstrap required, opened a new log
I20260811 00:16:22.073434 3209 ts_tablet_manager.cc:1397] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 00:16:22.074987 3204 raft_consensus.cc:348] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 } } peers { permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 } } peers { permanent_uuid: "b104e0a1d2034c489e89a15447205e59" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35975 } }
I20260811 00:16:22.075474 3204 raft_consensus.cc:374] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:16:22.075577 3204 raft_consensus.cc:729] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b104e0a1d2034c489e89a15447205e59, State: Initialized, Role: FOLLOWER
I20260811 00:16:22.076292 3204 consensus_queue.cc:260] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 } } peers { permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 } } peers { permanent_uuid: "b104e0a1d2034c489e89a15447205e59" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35975 } }
I20260811 00:16:22.081939 2941 heartbeater.cc:503] Master 127.7.94.62:40801 was elected leader, sending a full tablet report...
I20260811 00:16:22.082720 3204 ts_tablet_manager.cc:1428] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59: Time spent starting tablet: real 0.019s user 0.007s sys 0.010s
I20260811 00:16:22.083282 3209 raft_consensus.cc:348] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 } } peers { permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 } } peers { permanent_uuid: "b104e0a1d2034c489e89a15447205e59" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35975 } }
I20260811 00:16:22.083683 3209 raft_consensus.cc:374] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:16:22.083806 3209 raft_consensus.cc:729] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f39c6323ef42a8842b3d870b90ce2e, State: Initialized, Role: FOLLOWER
I20260811 00:16:22.084446 3209 consensus_queue.cc:260] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 } } peers { permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 } } peers { permanent_uuid: "b104e0a1d2034c489e89a15447205e59" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35975 } }
I20260811 00:16:22.084595 3207 raft_consensus.cc:348] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 } } peers { permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 } } peers { permanent_uuid: "b104e0a1d2034c489e89a15447205e59" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35975 } }
I20260811 00:16:22.085007 3207 raft_consensus.cc:374] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:16:22.085136 3207 raft_consensus.cc:729] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b77d2e865944dc0b522cf39c4aeeb92, State: Initialized, Role: FOLLOWER
I20260811 00:16:22.085866 3207 consensus_queue.cc:260] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 } } peers { permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 } } peers { permanent_uuid: "b104e0a1d2034c489e89a15447205e59" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35975 } }
I20260811 00:16:22.086344 3209 ts_tablet_manager.cc:1428] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e: Time spent starting tablet: real 0.013s user 0.010s sys 0.000s
I20260811 00:16:22.086994 3172 heartbeater.cc:503] Master 127.7.94.62:40801 was elected leader, sending a full tablet report...
I20260811 00:16:22.087657 3207 ts_tablet_manager.cc:1428] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
W20260811 00:16:22.142007 2942 tablet.cc:2369] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:16:22.191951 3174 tablet.cc:2369] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:16:22.219292 3214 raft_consensus.cc:482] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:16:22.219643 3214 raft_consensus.cc:504] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 } } peers { permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 } } peers { permanent_uuid: "b104e0a1d2034c489e89a15447205e59" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35975 } }
I20260811 00:16:22.221473 3214 leader_election.cc:288] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b77d2e865944dc0b522cf39c4aeeb92 (127.7.94.3:42797), 64f39c6323ef42a8842b3d870b90ce2e (127.7.94.1:42897)
I20260811 00:16:22.231583 3196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:22.221983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:43519 (local address 127.7.94.3:42797)
0811 00:16:22.222202 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:22.222209 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:22.223559 (+ 1350us) server_negotiation.cc:407] Connection header received
0811 00:16:22.223684 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:22.223690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:22.223769 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:22.223867 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:16:22.224780 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.225874 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.229094 (+ 3220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.230374 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.230512 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:22.230621 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:22.231202 (+ 581us) server_negotiation.cc:299] Negotiation successful
0811 00:16:22.231335 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:16:22.231587 3197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:22.222034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:41909 (local address 127.7.94.1:42897)
0811 00:16:22.222303 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:22.222307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:22.223482 (+ 1175us) server_negotiation.cc:407] Connection header received
0811 00:16:22.223695 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:22.223699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:22.223769 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:22.223868 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:16:22.224661 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.225838 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.229095 (+ 3257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.230374 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.230512 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:22.230621 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:22.231202 (+ 581us) server_negotiation.cc:299] Negotiation successful
0811 00:16:22.231335 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260811 00:16:22.231863 3221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:22.221779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:42897 (local address 127.7.94.2:41909)
0811 00:16:22.223357 (+ 1578us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:22.223378 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:16:22.223541 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:16:22.223922 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:16:22.223927 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:16:22.223976 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:16:22.224491 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:22.224511 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.225991 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.225997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:22.228799 (+ 2802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:22.228815 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.230668 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.230675 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:22.230974 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:22.231054 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 00:16:22.231157 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 00:16:22.231339 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1437,"mutex_wait_us":326,"spinlock_wait_cycles":384,"thread_start_us":1155,"threads_started":1}
I20260811 00:16:22.232196 3222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:22.221965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:42797 (local address 127.7.94.2:43519)
0811 00:16:22.223448 (+ 1483us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:22.223462 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 00:16:22.223556 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:16:22.223922 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:16:22.223927 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:16:22.223981 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:16:22.224605 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:22.224619 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.225991 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.225997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:22.228799 (+ 2802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:22.228815 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.230671 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.230676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:22.230978 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:22.231062 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 00:16:22.231157 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 00:16:22.231338 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1367,"mutex_wait_us":390,"spinlock_wait_cycles":38144,"thread_start_us":969,"threads_started":1}
I20260811 00:16:22.233042 2634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8efc7418abc4a18a7ef3793728afebe" candidate_uuid: "b104e0a1d2034c489e89a15447205e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f39c6323ef42a8842b3d870b90ce2e" is_pre_election: true
I20260811 00:16:22.233186 3100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8efc7418abc4a18a7ef3793728afebe" candidate_uuid: "b104e0a1d2034c489e89a15447205e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" is_pre_election: true
I20260811 00:16:22.233719 2634 raft_consensus.cc:2454] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b104e0a1d2034c489e89a15447205e59 in term 0.
I20260811 00:16:22.233763 3100 raft_consensus.cc:2454] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b104e0a1d2034c489e89a15447205e59 in term 0.
I20260811 00:16:22.234390 2759 leader_election.cc:302] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f39c6323ef42a8842b3d870b90ce2e, b104e0a1d2034c489e89a15447205e59; no voters:
I20260811 00:16:22.234824 3214 raft_consensus.cc:2781] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:16:22.234982 3214 raft_consensus.cc:482] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:16:22.235095 3214 raft_consensus.cc:3037] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:16:22.237335 3214 raft_consensus.cc:504] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 } } peers { permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 } } peers { permanent_uuid: "b104e0a1d2034c489e89a15447205e59" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35975 } }
I20260811 00:16:22.237969 3214 leader_election.cc:288] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [CANDIDATE]: Term 1 election: Requested vote from peers 2b77d2e865944dc0b522cf39c4aeeb92 (127.7.94.3:42797), 64f39c6323ef42a8842b3d870b90ce2e (127.7.94.1:42897)
I20260811 00:16:22.238379 3100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8efc7418abc4a18a7ef3793728afebe" candidate_uuid: "b104e0a1d2034c489e89a15447205e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b77d2e865944dc0b522cf39c4aeeb92"
I20260811 00:16:22.238471 2634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8efc7418abc4a18a7ef3793728afebe" candidate_uuid: "b104e0a1d2034c489e89a15447205e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f39c6323ef42a8842b3d870b90ce2e"
I20260811 00:16:22.238606 3100 raft_consensus.cc:3037] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:16:22.238716 2634 raft_consensus.cc:3037] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:16:22.240816 3100 raft_consensus.cc:2454] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b104e0a1d2034c489e89a15447205e59 in term 1.
I20260811 00:16:22.240825 2634 raft_consensus.cc:2454] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b104e0a1d2034c489e89a15447205e59 in term 1.
I20260811 00:16:22.241202 2759 leader_election.cc:302] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f39c6323ef42a8842b3d870b90ce2e, b104e0a1d2034c489e89a15447205e59; no voters:
I20260811 00:16:22.241503 3214 raft_consensus.cc:2781] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:16:22.242080 3214 raft_consensus.cc:686] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [term 1 LEADER]: Becoming Leader. State: Replica: b104e0a1d2034c489e89a15447205e59, State: Running, Role: LEADER
I20260811 00:16:22.242555 3214 consensus_queue.cc:237] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 } } peers { permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 } } peers { permanent_uuid: "b104e0a1d2034c489e89a15447205e59" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35975 } }
I20260811 00:16:22.247375 2385 catalog_manager.cc:5495] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 reported cstate change: term changed from 0 to 1, leader changed from <none> to b104e0a1d2034c489e89a15447205e59 (127.7.94.2). New cstate: current_term: 1 leader_uuid: "b104e0a1d2034c489e89a15447205e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b104e0a1d2034c489e89a15447205e59" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35975 } health_report { overall_health: HEALTHY } } }
I20260811 00:16:22.276412 3200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:22.270519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37394 (local address 127.7.94.2:35975)
0811 00:16:22.270726 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:22.270733 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:22.270800 (+ 67us) server_negotiation.cc:407] Connection header received
0811 00:16:22.271080 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:22.271084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:22.271172 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:22.271326 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:16:22.271755 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.272579 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.274192 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.274593 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.274734 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:22.275284 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:16:22.275823 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:16:22.275984 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:22.276036 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 00:16:22.276130 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:16:22.284770 3100 raft_consensus.cc:1261] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92 [term 1 FOLLOWER]: Refusing update from remote peer b104e0a1d2034c489e89a15447205e59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:16:22.284792 2634 raft_consensus.cc:1261] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e [term 1 FOLLOWER]: Refusing update from remote peer b104e0a1d2034c489e89a15447205e59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:16:22.285750 3223 consensus_queue.cc:1035] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:16:22.286123 3214 consensus_queue.cc:1035] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:16:22.297719 3229 mvcc.cc:205] Tried to move back new op lower bound from 7317124637828521984 to 7317124637669302272. Current Snapshot: MvccSnapshot[applied={T|T < 7317124637828521984}]
I20260811 00:16:22.301450 3231 mvcc.cc:205] Tried to move back new op lower bound from 7317124637828521984 to 7317124637669302272. Current Snapshot: MvccSnapshot[applied={T|T < 7317124637828521984}]
I20260811 00:16:22.311510 2385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:16:22.311767 2385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:16:22.315312 2385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22: Sending DeleteTablet for 3 replicas of tablet e8efc7418abc4a18a7ef3793728afebe
I20260811 00:16:22.316457 2597 tablet_service.cc:1514] Processing DeleteTablet for tablet e8efc7418abc4a18a7ef3793728afebe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:16:22 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:34570
I20260811 00:16:22.316592 3064 tablet_service.cc:1514] Processing DeleteTablet for tablet e8efc7418abc4a18a7ef3793728afebe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:16:22 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:55104
I20260811 00:16:22.316710 2830 tablet_service.cc:1514] Processing DeleteTablet for tablet e8efc7418abc4a18a7ef3793728afebe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:16:22 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:37382
I20260811 00:16:22.317368 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 2491
I20260811 00:16:22.329589 3245 tablet_replica.cc:330] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92: stopping tablet replica
I20260811 00:16:22.330369 3247 tablet_replica.cc:330] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59: stopping tablet replica
I20260811 00:16:22.330986 3247 raft_consensus.cc:2229] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:16:22.331719 3244 tablet_replica.cc:330] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e: stopping tablet replica
I20260811 00:16:22.331831 3247 raft_consensus.cc:2258] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:16:22.332203 3244 raft_consensus.cc:2229] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:16:22.332613 3245 raft_consensus.cc:2229] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:16:22.332851 3244 raft_consensus.cc:2258] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:16:22.333081 3245 raft_consensus.cc:2258] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:16:22.335047 3244 ts_tablet_manager.cc:1905] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:16:22.335780 3245 ts_tablet_manager.cc:1905] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:16:22.342799 3245 ts_tablet_manager.cc:1918] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:16:22.342976 3245 log.cc:1196] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/e8efc7418abc4a18a7ef3793728afebe
I20260811 00:16:22.342960 3244 ts_tablet_manager.cc:1918] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:16:22.343124 3244 log.cc:1196] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/e8efc7418abc4a18a7ef3793728afebe
I20260811 00:16:22.343593 3245 ts_tablet_manager.cc:1939] T e8efc7418abc4a18a7ef3793728afebe P 2b77d2e865944dc0b522cf39c4aeeb92: Deleting consensus metadata
I20260811 00:16:22.343636 3244 ts_tablet_manager.cc:1939] T e8efc7418abc4a18a7ef3793728afebe P 64f39c6323ef42a8842b3d870b90ce2e: Deleting consensus metadata
Aug 11 00:16:22 dist-test-slave-8hhm krb5kdc[2318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:16:22.345466 2365 catalog_manager.cc:4841] TS 64f39c6323ef42a8842b3d870b90ce2e (127.7.94.1:42897): tablet e8efc7418abc4a18a7ef3793728afebe (table test-table [id=e2096dff9f904239bd97d9946e5f9133]) successfully deleted
I20260811 00:16:22.345548 2366 catalog_manager.cc:4841] TS 2b77d2e865944dc0b522cf39c4aeeb92 (127.7.94.3:42797): tablet e8efc7418abc4a18a7ef3793728afebe (table test-table [id=e2096dff9f904239bd97d9946e5f9133]) successfully deleted
I20260811 00:16:22.347363 3247 ts_tablet_manager.cc:1905] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:16:22.352798 3247 ts_tablet_manager.cc:1918] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:16:22.352993 3247 log.cc:1196] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/e8efc7418abc4a18a7ef3793728afebe
I20260811 00:16:22.353641 3247 ts_tablet_manager.cc:1939] T e8efc7418abc4a18a7ef3793728afebe P b104e0a1d2034c489e89a15447205e59: Deleting consensus metadata
I20260811 00:16:22.354403 3197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:22.318768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34574 (local address 127.7.94.1:42897)
0811 00:16:22.318991 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:22.319000 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:22.329280 (+ 10280us) server_negotiation.cc:407] Connection header received
0811 00:16:22.333549 (+ 4269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:22.333554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:22.333640 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:22.333764 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:22.336037 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.337277 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.339337 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.340084 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.340193 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:22.346559 (+ 6366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:22.346573 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:22.346592 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:22.346633 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:22.350585 (+ 3952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:22.351556 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:22.351565 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:22.351569 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:22.351753 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:22.352281 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:22.352286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:22.352290 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:22.352748 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:22.353042 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:22.353766 (+ 724us) server_negotiation.cc:299] Negotiation successful
0811 00:16:22.354003 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 00:16:22.355108 2709 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:16:22.355880 2366 catalog_manager.cc:4841] TS b104e0a1d2034c489e89a15447205e59 (127.7.94.2:35975): tablet e8efc7418abc4a18a7ef3793728afebe (table test-table [id=e2096dff9f904239bd97d9946e5f9133]) successfully deleted
I20260811 00:16:22.484862 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 2491
I20260811 00:16:22.514089 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 2729
Aug 11 00:16:22 dist-test-slave-8hhm krb5kdc[2318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.2@KRBTEST.COM
I20260811 00:16:22.532619 3200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:22.514944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37402 (local address 127.7.94.2:35975)
0811 00:16:22.515144 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:22.515152 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:22.515194 (+ 42us) server_negotiation.cc:407] Connection header received
0811 00:16:22.515376 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:22.515382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:22.515465 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:22.515594 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:22.517037 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.518274 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.519789 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.520505 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.520634 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:22.525361 (+ 4727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:22.525380 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:22.525401 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:22.525449 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:22.529718 (+ 4269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:22.530437 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:22.530450 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:22.530454 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:22.530614 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:22.530941 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:22.530946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:22.530949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:22.531437 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:22.531824 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:22.531982 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 00:16:22.532220 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 00:16:22.533633 2933 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:16:22.649657 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 2729
I20260811 00:16:22.682227 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 2948
Aug 11 00:16:22 dist-test-slave-8hhm krb5kdc[2318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.3@KRBTEST.COM
I20260811 00:16:22.697602 3196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:22.683042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55112 (local address 127.7.94.3:42797)
0811 00:16:22.683207 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:22.683211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:22.683230 (+ 19us) server_negotiation.cc:407] Connection header received
0811 00:16:22.683390 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:22.683393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:22.683469 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:22.683570 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:22.685143 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.685927 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.687157 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:22.687545 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:22.687629 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:22.691998 (+ 4369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:22.692013 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:22.692029 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:22.692060 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:22.695067 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:22.695780 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:22.695787 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:22.695790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:22.695925 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:22.696261 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:22.696264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:22.696266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:22.696585 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:22.696808 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:22.697056 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 00:16:22.697263 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 00:16:22.698292 3159 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:16:22.787446 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 2948
I20260811 00:16:22.814239 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 2340
I20260811 00:16:22.815476 2435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:16:22.930243 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 2340
I20260811 00:16:42.965157 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:40801
--webserver_interface=127.7.94.62
--webserver_port=46187
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:40801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:16:43.315593 3257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:16:43.315886 3257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:16:43.316049 3257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:16:43.323428 3257 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 00:16:43.323534 3257 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 00:16:43.323608 3257 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 00:16:43.323684 3257 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:16:43.323781 3257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:16:43.323908 3257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:16:43.323987 3257 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:16:43.324072 3257 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:16:43.334077 3257 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:40801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:40801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=46187
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:16:43.335439 3257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:16:43.336791 3257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:16:43.362867 3266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:16:43.363200 3257 server_base.cc:1034] running on GCE node
W20260811 00:16:43.362805 3271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:43.362892 3267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:16:43.364446 3257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:16:43.366004 3257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:16:43.367247 3257 hybrid_clock.cc:648] HybridClock initialized: now 1786407403367158 us; error 54 us; skew 500 ppm
I20260811 00:16:43.367658 3257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:16:43 dist-test-slave-8hhm krb5kdc[2318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407403, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:16:43.375840 3257 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:16:43.377689 3257 webserver.cc:460] Webserver started at http://127.7.94.62:46187/ using document root <none> and password file <none>
I20260811 00:16:43.378547 3257 fs_manager.cc:359] Metadata directory not provided
I20260811 00:16:43.378701 3257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:16:43.384930 3257 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 00:16:43.387975 3280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:43.389554 3257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:16:43.389755 3257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "4d5f62bf2ddd4264b716ec18b3554d22"
format_stamp: "Formatted at 2026-08-11 00:16:20 on dist-test-slave-8hhm"
server_key: "9c3059d80090978456b5302472bd99c1"
server_key_iv: "6c5f2409687018dc568f8e554105130a"
server_key_version: "encryptionkey@0"
I20260811 00:16:43.390854 3257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:16:43.415401 3257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:16:43.419914 3257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:16:43.420236 3257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:16:43.439888 3257 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:40801
I20260811 00:16:43.439918 3364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:40801 every 8 connection(s)
I20260811 00:16:43.441499 3257 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:16:43.448935 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 3257
I20260811 00:16:43.449009 3366 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:16:43.450098 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:42897
--local_ip_for_outbound_sockets=127.7.94.1
--tserver_master_addrs=127.7.94.62:40801
--webserver_port=35333
--webserver_interface=127.7.94.1
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:16:43.454494 3366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22: Bootstrap starting.
I20260811 00:16:43.477738 3366 log.cc:824] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22: Log is configured to *not* fsync() on all Append() calls
I20260811 00:16:43.490092 3366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:16:43.490674 3366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22: Bootstrap complete.
I20260811 00:16:43.500337 3366 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40801 } }
I20260811 00:16:43.501160 3366 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d5f62bf2ddd4264b716ec18b3554d22, State: Initialized, Role: FOLLOWER
I20260811 00:16:43.501915 3366 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40801 } }
I20260811 00:16:43.502173 3366 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:16:43.502291 3366 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:16:43.502482 3366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 1 FOLLOWER]: Advancing to term 2
I20260811 00:16:43.506428 3366 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40801 } }
I20260811 00:16:43.506961 3366 leader_election.cc:302] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d5f62bf2ddd4264b716ec18b3554d22; no voters:
I20260811 00:16:43.507665 3366 leader_election.cc:288] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 00:16:43.507797 3370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 00:16:43.509248 3370 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 2 LEADER]: Becoming Leader. State: Replica: 4d5f62bf2ddd4264b716ec18b3554d22, State: Running, Role: LEADER
I20260811 00:16:43.509855 3370 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40801 } }
I20260811 00:16:43.510146 3366 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:16:43.512563 3375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d5f62bf2ddd4264b716ec18b3554d22. Latest consensus state: current_term: 2 leader_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40801 } } }
I20260811 00:16:43.513000 3375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [sys.catalog]: This master's current role is: LEADER
I20260811 00:16:43.513537 3373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40801 } } }
I20260811 00:16:43.514797 3373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [sys.catalog]: This master's current role is: LEADER
I20260811 00:16:43.524137 3382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:16:43.532238 3382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:16:43.533631 3382 catalog_manager.cc:1259] Loaded cluster ID: 5eaf0bddeab44da78a5fcacb99d6b83f
I20260811 00:16:43.533725 3382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:16:43.537942 3382 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:16:43.539606 3382 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22: Loaded TSK: 0
I20260811 00:16:43.568118 3382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22: Generated new TSK 1
I20260811 00:16:43.568538 3382 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 00:16:43.818048 3368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:16:43.818322 3368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:16:43.818472 3368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:16:43.825886 3368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:16:43.826014 3368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:16:43.826208 3368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:16:43.836090 3368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:42897
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=35333
--tserver_master_addrs=127.7.94.62:40801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:16:43.837406 3368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:16:43.838766 3368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:16:43.853943 3409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:43.854146 3410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:16:43.854938 3368 server_base.cc:1034] running on GCE node
W20260811 00:16:43.855088 3413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:16:43.855938 3368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:16:43.857167 3368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:16:43.858457 3368 hybrid_clock.cc:648] HybridClock initialized: now 1786407403858404 us; error 39 us; skew 500 ppm
I20260811 00:16:43.858960 3368 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:16:43 dist-test-slave-8hhm krb5kdc[2318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407403, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:16:43.863116 3368 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:16:43.864877 3368 webserver.cc:460] Webserver started at http://127.7.94.1:35333/ using document root <none> and password file <none>
I20260811 00:16:43.865813 3368 fs_manager.cc:359] Metadata directory not provided
I20260811 00:16:43.865973 3368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:16:43.873040 3368 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 00:16:43.876547 3428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:43.878108 3368 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:16:43.878296 3368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "64f39c6323ef42a8842b3d870b90ce2e"
format_stamp: "Formatted at 2026-08-11 00:16:20 on dist-test-slave-8hhm"
server_key: "b21b4bf7975842607252b11e3eeb3642"
server_key_iv: "8c91d7630c72243ae736ed371d510efd"
server_key_version: "encryptionkey@0"
I20260811 00:16:43.879310 3368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:16:43.906270 3368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:16:43.910879 3368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:16:43.911206 3368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:16:43.912276 3368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:16:43.914145 3368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:16:43.914276 3368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:43.914412 3368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:16:43.914523 3368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:43.948192 3368 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:42897
I20260811 00:16:43.948249 3624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:42897 every 8 connection(s)
I20260811 00:16:43.949710 3368 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:16:43.953768 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 3368
I20260811 00:16:43.954495 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:35975
--local_ip_for_outbound_sockets=127.7.94.2
--tserver_master_addrs=127.7.94.62:40801
--webserver_port=43887
--webserver_interface=127.7.94.2
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:16:43 dist-test-slave-8hhm krb5kdc[2318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407403, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:16:43.976759 3631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:43.952826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:49947 (local address 127.7.94.62:40801)
0811 00:16:43.953566 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:43.953588 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:43.954423 (+ 835us) server_negotiation.cc:407] Connection header received
0811 00:16:43.957209 (+ 2786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:43.957251 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:43.957722 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:43.958140 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:43.959688 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:43.961653 (+ 1965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:43.963274 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:43.964422 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:43.964627 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:43.968698 (+ 4071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:43.968707 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:43.968723 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:43.968760 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:43.971958 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:43.972942 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:43.972949 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:43.972955 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:43.973122 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:43.973663 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:43.973667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:43.973669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:43.974129 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:43.974281 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:43.975307 (+ 1026us) server_negotiation.cc:299] Negotiation successful
0811 00:16:43.975633 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":278,"threads_started":1}
I20260811 00:16:43.977190 3634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:43.952905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.1:49947)
0811 00:16:43.954216 (+ 1311us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:43.954270 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 00:16:43.956571 (+ 2301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:16:43.958298 (+ 1727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:16:43.958306 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:16:43.958854 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:16:43.959469 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:43.959490 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:43.961830 (+ 2340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:43.961834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:43.963116 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:43.963130 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:43.964651 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:43.964658 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:43.964862 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:43.965654 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:16:43.965677 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:16:43.968380 (+ 2703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:16:43.972107 (+ 3727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:16:43.972119 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:16:43.972138 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:16:43.972804 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:16:43.973280 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:16:43.973288 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:16:43.973295 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:16:43.973553 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:16:43.974294 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:16:43.974305 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:16:43.974797 (+ 492us) client_negotiation.cc:769] Sending connection context
0811 00:16:43.975127 (+ 330us) client_negotiation.cc:240] Negotiation successful
0811 00:16:43.975726 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1027,"thread_start_us":800,"threads_started":1}
I20260811 00:16:43.980180 3627 heartbeater.cc:347] Connected to a master server at 127.7.94.62:40801
I20260811 00:16:43.980504 3627 heartbeater.cc:464] Registering TS with master...
I20260811 00:16:43.981384 3627 heartbeater.cc:511] Master 127.7.94.62:40801 requested a full tablet report, sending...
I20260811 00:16:43.983724 3304 ts_manager.cc:194] Registered new tserver with Master: 64f39c6323ef42a8842b3d870b90ce2e (127.7.94.1:42897)
I20260811 00:16:43.985309 3304 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:49947
W20260811 00:16:44.304440 3636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:16:44.304811 3636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:16:44.304997 3636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:16:44.312744 3636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:16:44.312901 3636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:16:44.313081 3636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:16:44.323275 3636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:35975
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=43887
--tserver_master_addrs=127.7.94.62:40801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:16:44.324643 3636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:16:44.326063 3636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:16:44.339440 3643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:44.339381 3647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:44.339429 3644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:16:44.340415 3636 server_base.cc:1034] running on GCE node
I20260811 00:16:44.341112 3636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:16:44.342089 3636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:16:44.343317 3636 hybrid_clock.cc:648] HybridClock initialized: now 1786407404343275 us; error 37 us; skew 500 ppm
I20260811 00:16:44.343657 3636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:16:44 dist-test-slave-8hhm krb5kdc[2318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407404, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:16:44.346848 3636 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:16:44.348352 3636 webserver.cc:460] Webserver started at http://127.7.94.2:43887/ using document root <none> and password file <none>
I20260811 00:16:44.349185 3636 fs_manager.cc:359] Metadata directory not provided
I20260811 00:16:44.349373 3636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:16:44.354812 3636 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:16:44.357574 3656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:44.359000 3636 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:16:44.359171 3636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "b104e0a1d2034c489e89a15447205e59"
format_stamp: "Formatted at 2026-08-11 00:16:21 on dist-test-slave-8hhm"
server_key: "51f96b77b91813a53e5180888c06ebb8"
server_key_iv: "c3d653e05b04f6e31eb669ff31578dec"
server_key_version: "encryptionkey@0"
I20260811 00:16:44.360157 3636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:16:44.384193 3636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:16:44.388386 3636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:16:44.388713 3636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:16:44.389822 3636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:16:44.391238 3636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:16:44.391367 3636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:44.391498 3636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:16:44.391588 3636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:44.427122 3636 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:35975
I20260811 00:16:44.427218 3845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:35975 every 8 connection(s)
I20260811 00:16:44.428689 3636 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:16:44.438968 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 3636
I20260811 00:16:44.439729 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:42797
--local_ip_for_outbound_sockets=127.7.94.3
--tserver_master_addrs=127.7.94.62:40801
--webserver_port=45553
--webserver_interface=127.7.94.3
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 00:16:44 dist-test-slave-8hhm krb5kdc[2318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407404, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:16:44.452147 3631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:44.431398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:39133 (local address 127.7.94.62:40801)
0811 00:16:44.431577 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:44.431582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:44.432833 (+ 1251us) server_negotiation.cc:407] Connection header received
0811 00:16:44.434777 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:44.434782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:44.434854 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:44.434948 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:44.436608 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:44.437397 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:44.438998 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:44.439697 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:44.439827 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:44.444709 (+ 4882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:44.444717 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:44.444722 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:44.444770 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:44.448372 (+ 3602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:44.449352 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:44.449358 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:44.449362 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:44.449503 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:44.450029 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:44.450033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:44.450037 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:44.450417 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:44.450554 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:44.451487 (+ 933us) server_negotiation.cc:299] Negotiation successful
0811 00:16:44.451754 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:16:44.453517 3851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:44.431929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.2:39133)
0811 00:16:44.432660 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:44.432706 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 00:16:44.434471 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:16:44.435260 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:16:44.435272 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:16:44.435869 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:16:44.436447 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:44.436465 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:44.437541 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:44.437545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:44.438843 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:44.438855 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:44.439956 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:44.439967 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:44.440316 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:44.441149 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:16:44.441193 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:16:44.444423 (+ 3230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:16:44.448530 (+ 4107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:16:44.448542 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:16:44.448561 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:16:44.449246 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:16:44.449623 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:16:44.449631 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:16:44.449637 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:16:44.449910 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:16:44.450554 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:16:44.450564 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:16:44.451035 (+ 471us) client_negotiation.cc:769] Sending connection context
0811 00:16:44.451372 (+ 337us) client_negotiation.cc:240] Negotiation successful
0811 00:16:44.451976 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":285,"threads_started":1}
I20260811 00:16:44.455139 3848 heartbeater.cc:347] Connected to a master server at 127.7.94.62:40801
I20260811 00:16:44.455477 3848 heartbeater.cc:464] Registering TS with master...
I20260811 00:16:44.456353 3848 heartbeater.cc:511] Master 127.7.94.62:40801 requested a full tablet report, sending...
I20260811 00:16:44.458170 3304 ts_manager.cc:194] Registered new tserver with Master: b104e0a1d2034c489e89a15447205e59 (127.7.94.2:35975)
I20260811 00:16:44.458983 3304 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:39133
W20260811 00:16:44.794088 3859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:16:44.794450 3859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:16:44.794648 3859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:16:44.802278 3859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:16:44.802449 3859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:16:44.802624 3859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:16:44.813185 3859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:42797
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=45553
--tserver_master_addrs=127.7.94.62:40801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:16:44.814602 3859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:16:44.815982 3859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:16:44.829564 3869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:44.829528 3871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:16:44.829628 3868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:16:44.830454 3859 server_base.cc:1034] running on GCE node
I20260811 00:16:44.831025 3859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:16:44.831943 3859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:16:44.833141 3859 hybrid_clock.cc:648] HybridClock initialized: now 1786407404833078 us; error 65 us; skew 500 ppm
I20260811 00:16:44.833501 3859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:16:44 dist-test-slave-8hhm krb5kdc[2318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407404, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:16:44.836982 3859 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:16:44.838622 3859 webserver.cc:460] Webserver started at http://127.7.94.3:45553/ using document root <none> and password file <none>
I20260811 00:16:44.839375 3859 fs_manager.cc:359] Metadata directory not provided
I20260811 00:16:44.839540 3859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:16:44.845361 3859 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 00:16:44.848115 3880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:44.849577 3859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:16:44.849763 3859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "2b77d2e865944dc0b522cf39c4aeeb92"
format_stamp: "Formatted at 2026-08-11 00:16:21 on dist-test-slave-8hhm"
server_key: "b9f16ee39b6288354622e48e5f3fb461"
server_key_iv: "f20c3d7ec890349495f7699d9f6fdfbd"
server_key_version: "encryptionkey@0"
I20260811 00:16:44.850780 3859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:16:44.867370 3859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:16:44.871420 3859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:16:44.871721 3859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:16:44.872746 3859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:16:44.874270 3859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:16:44.874418 3859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:44.874559 3859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:16:44.874670 3859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:16:44.921128 3859 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:42797
I20260811 00:16:44.921275 4068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:42797 every 8 connection(s)
I20260811 00:16:44.922667 3859 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:16:44.925670 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 3859
Aug 11 00:16:44 dist-test-slave-8hhm krb5kdc[2318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407404, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:16:44.944468 4075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:44.926665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56176 (local address 127.7.94.62:40801)
0811 00:16:44.930026 (+ 3361us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:44.930032 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:44.930049 (+ 17us) server_negotiation.cc:407] Connection header received
0811 00:16:44.930095 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:44.930099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:44.930167 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:44.930280 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:44.931980 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:44.932778 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:44.934445 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:44.934864 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:44.934990 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:44.938301 (+ 3311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:44.938305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:44.938309 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:44.938339 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:44.941452 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:44.942217 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:44.942223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:44.942230 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:44.942408 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:44.942854 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:44.942859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:44.942862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:44.943216 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:44.943446 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:44.943826 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 00:16:44.944050 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3242,"thread_start_us":3119,"threads_started":1}
I20260811 00:16:44.945318 3631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:44.925434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:35263 (local address 127.7.94.62:40801)
0811 00:16:44.925664 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:44.925671 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:44.927001 (+ 1330us) server_negotiation.cc:407] Connection header received
0811 00:16:44.928912 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:44.928917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:44.929008 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:44.929140 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:44.931198 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:44.931945 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:44.933637 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:44.934076 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:44.934214 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:44.938556 (+ 4342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:44.938560 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:44.938563 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:44.938615 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:44.941631 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:44.942500 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:44.942506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:44.942509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:44.942675 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:44.943228 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:44.943231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:44.943233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:44.943622 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:44.943759 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:44.944720 (+ 961us) server_negotiation.cc:299] Negotiation successful
0811 00:16:44.944936 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:16:44.946708 4072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:44.925970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.3:35263)
0811 00:16:44.926815 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:44.926869 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 00:16:44.928644 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:16:44.929636 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:16:44.929650 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:16:44.930444 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:16:44.931027 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:44.931043 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:44.932135 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:44.932144 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:44.933489 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:44.933502 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:44.934310 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:44.934319 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:44.934560 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:44.935310 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:16:44.935340 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:16:44.938341 (+ 3001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:16:44.941782 (+ 3441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:16:44.941788 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:16:44.941801 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:16:44.942328 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:16:44.942817 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:16:44.942829 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:16:44.942835 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:16:44.943088 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:16:44.943758 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:16:44.943766 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:16:44.944227 (+ 461us) client_negotiation.cc:769] Sending connection context
0811 00:16:44.944540 (+ 313us) client_negotiation.cc:240] Negotiation successful
0811 00:16:44.945468 (+ 928us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":364,"threads_started":1}
I20260811 00:16:44.948310 4069 heartbeater.cc:347] Connected to a master server at 127.7.94.62:40801
I20260811 00:16:44.948621 4069 heartbeater.cc:464] Registering TS with master...
I20260811 00:16:44.949419 4069 heartbeater.cc:511] Master 127.7.94.62:40801 requested a full tablet report, sending...
I20260811 00:16:44.950873 3304 ts_manager.cc:194] Registered new tserver with Master: 2b77d2e865944dc0b522cf39c4aeeb92 (127.7.94.3:42797)
I20260811 00:16:44.951988 3304 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:35263
I20260811 00:16:44.959479 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:16:44.966150 3631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:44.960381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56192 (local address 127.7.94.62:40801)
0811 00:16:44.960540 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:44.960544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:44.961105 (+ 561us) server_negotiation.cc:407] Connection header received
0811 00:16:44.961382 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:44.961386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:44.961527 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:44.961669 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:16:44.962106 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:44.962816 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:44.964301 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:44.964715 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:44.964824 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:44.965372 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:16:44.965672 (+ 300us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 00:16:44.965959 (+ 287us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56192: token expired
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 00:16:44.966311 3631 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56192: token expired
W20260811 00:16:44.966920 3182 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.94.62:40801)
I20260811 00:16:44.980715 3631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:44.967466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56202 (local address 127.7.94.62:40801)
0811 00:16:44.967633 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:44.967639 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:44.967739 (+ 100us) server_negotiation.cc:407] Connection header received
0811 00:16:44.967933 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:44.967942 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:44.968000 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:44.968118 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:44.969597 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:44.970309 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:44.971795 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:44.972202 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:44.972339 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:44.975202 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:44.975207 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:44.975211 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:44.975245 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:44.977923 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:44.978610 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:44.978618 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:44.978623 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:44.978753 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:44.979182 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:44.979187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:44.979191 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:44.979503 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:44.979789 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:44.980133 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 00:16:44.980357 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 00:16:44.988569 3627 heartbeater.cc:503] Master 127.7.94.62:40801 was elected leader, sending a full tablet report...
I20260811 00:16:44.984392 3304 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56202:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:16:44.992420 3304 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:16:45.016631 4085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.004622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:42897 (local address 127.0.0.1:44850)
0811 00:16:45.005555 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:45.005598 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 00:16:45.005790 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:16:45.007211 (+ 1421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:16:45.007219 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:16:45.007252 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:16:45.007722 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:45.007743 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.010249 (+ 2506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.010258 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:45.013791 (+ 3533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:45.013813 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.015676 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.015683 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:45.015897 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:45.015951 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 00:16:45.016117 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 00:16:45.016310 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":76,"thread_start_us":480,"threads_started":1}
I20260811 00:16:45.016682 4088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.005315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44850 (local address 127.7.94.1:42897)
0811 00:16:45.006530 (+ 1215us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:45.006536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:45.006560 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:16:45.006642 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:45.006648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:45.006930 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:45.008010 (+ 1080us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:16:45.008303 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.010052 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.014011 (+ 3959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.015459 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.015604 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:45.015884 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:45.016182 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 00:16:45.016355 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1053,"thread_start_us":921,"threads_started":1}
I20260811 00:16:45.018221 4086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.004888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:35975 (local address 127.0.0.1:60672)
0811 00:16:45.006848 (+ 1960us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:45.006871 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:16:45.007071 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:16:45.008178 (+ 1107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:16:45.008185 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:16:45.008219 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:16:45.008701 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:45.008715 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.011237 (+ 2522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.011244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:45.014321 (+ 3077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:45.014335 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.017237 (+ 2902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.017247 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:45.017449 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:45.017496 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 00:16:45.017601 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 00:16:45.017759 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1811,"mutex_wait_us":3,"spinlock_wait_cycles":4992,"thread_start_us":1790,"threads_started":1}
I20260811 00:16:45.019076 4089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.005586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60672 (local address 127.7.94.2:35975)
0811 00:16:45.006377 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:45.006385 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:45.006972 (+ 587us) server_negotiation.cc:407] Connection header received
0811 00:16:45.007307 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:45.007317 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:45.007596 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:45.007796 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:16:45.008931 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.010983 (+ 2052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.014527 (+ 3544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.016008 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.016143 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:45.016335 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:45.018559 (+ 2224us) server_negotiation.cc:299] Negotiation successful
0811 00:16:45.018741 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":516,"threads_started":1}
I20260811 00:16:45.020896 3513 tablet_service.cc:1467] Processing CreateTablet for tablet 47ba39d799314e7cb94e7ac7acd19cd3 (DEFAULT_TABLE table=test-table [id=52f1ca93cfa04456812658887ab740b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:16:45.022094 4092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.007443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52258 (local address 127.7.94.3:42797)
0811 00:16:45.012473 (+ 5030us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:45.012480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:45.014281 (+ 1801us) server_negotiation.cc:407] Connection header received
0811 00:16:45.014435 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:45.014445 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:45.014697 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:45.014907 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:16:45.015494 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.017134 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.019749 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.020983 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.021136 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:45.021400 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:45.021612 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 00:16:45.021774 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4862,"thread_start_us":4712,"threads_started":1}
I20260811 00:16:45.022114 4091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.007053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:42797 (local address 127.0.0.1:52258)
0811 00:16:45.014130 (+ 7077us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:45.014149 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:16:45.014287 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:16:45.014917 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:16:45.014923 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:16:45.014948 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:16:45.015300 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:45.015311 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.017371 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.017376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:45.019576 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:45.019588 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.021248 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.021256 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:45.021413 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:45.021452 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 00:16:45.021591 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 00:16:45.021733 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6959,"mutex_wait_us":2,"thread_start_us":6810,"threads_started":1}
I20260811 00:16:45.023029 3513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47ba39d799314e7cb94e7ac7acd19cd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:16:45.023211 3735 tablet_service.cc:1467] Processing CreateTablet for tablet 47ba39d799314e7cb94e7ac7acd19cd3 (DEFAULT_TABLE table=test-table [id=52f1ca93cfa04456812658887ab740b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:16:45.024777 3735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47ba39d799314e7cb94e7ac7acd19cd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:16:45.026235 3967 tablet_service.cc:1467] Processing CreateTablet for tablet 47ba39d799314e7cb94e7ac7acd19cd3 (DEFAULT_TABLE table=test-table [id=52f1ca93cfa04456812658887ab740b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:16:45.027618 3967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47ba39d799314e7cb94e7ac7acd19cd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:16:45.035689 4097 tablet_bootstrap.cc:492] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e: Bootstrap starting.
I20260811 00:16:45.039867 4097 tablet_bootstrap.cc:654] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e: Neither blocks nor log segments found. Creating new log.
I20260811 00:16:45.041055 4098 tablet_bootstrap.cc:492] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59: Bootstrap starting.
I20260811 00:16:45.041234 4097 log.cc:824] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e: Log is configured to *not* fsync() on all Append() calls
I20260811 00:16:45.043763 4101 tablet_bootstrap.cc:492] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92: Bootstrap starting.
I20260811 00:16:45.044689 4097 tablet_bootstrap.cc:492] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e: No bootstrap required, opened a new log
I20260811 00:16:45.044822 4098 tablet_bootstrap.cc:654] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59: Neither blocks nor log segments found. Creating new log.
I20260811 00:16:45.044972 4097 ts_tablet_manager.cc:1397] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 00:16:45.046164 4098 log.cc:824] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59: Log is configured to *not* fsync() on all Append() calls
I20260811 00:16:45.049621 4101 tablet_bootstrap.cc:654] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92: Neither blocks nor log segments found. Creating new log.
I20260811 00:16:45.051600 4101 log.cc:824] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92: Log is configured to *not* fsync() on all Append() calls
I20260811 00:16:45.061041 4097 raft_consensus.cc:348] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 } } peers { permanent_uuid: "b104e0a1d2034c489e89a15447205e59" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35975 } } peers { permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 } }
I20260811 00:16:45.061688 4097 raft_consensus.cc:374] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:16:45.061798 4097 raft_consensus.cc:729] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f39c6323ef42a8842b3d870b90ce2e, State: Initialized, Role: FOLLOWER
I20260811 00:16:45.062561 4097 consensus_queue.cc:260] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 } } peers { permanent_uuid: "b104e0a1d2034c489e89a15447205e59" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35975 } } peers { permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 } }
I20260811 00:16:45.063825 4101 tablet_bootstrap.cc:492] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92: No bootstrap required, opened a new log
I20260811 00:16:45.064158 4101 ts_tablet_manager.cc:1397] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.006s
I20260811 00:16:45.064471 4098 tablet_bootstrap.cc:492] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59: No bootstrap required, opened a new log
I20260811 00:16:45.064764 4098 ts_tablet_manager.cc:1397] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.018s
I20260811 00:16:45.068331 4097 ts_tablet_manager.cc:1428] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e: Time spent starting tablet: real 0.023s user 0.012s sys 0.007s
I20260811 00:16:45.072880 4101 raft_consensus.cc:348] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 } } peers { permanent_uuid: "b104e0a1d2034c489e89a15447205e59" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35975 } } peers { permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 } }
I20260811 00:16:45.073413 4101 raft_consensus.cc:374] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:16:45.073531 4101 raft_consensus.cc:729] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b77d2e865944dc0b522cf39c4aeeb92, State: Initialized, Role: FOLLOWER
I20260811 00:16:45.074111 4101 consensus_queue.cc:260] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 } } peers { permanent_uuid: "b104e0a1d2034c489e89a15447205e59" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35975 } } peers { permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 } }
I20260811 00:16:45.074563 4098 raft_consensus.cc:348] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 } } peers { permanent_uuid: "b104e0a1d2034c489e89a15447205e59" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35975 } } peers { permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 } }
I20260811 00:16:45.074961 4098 raft_consensus.cc:374] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:16:45.075086 4098 raft_consensus.cc:729] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b104e0a1d2034c489e89a15447205e59, State: Initialized, Role: FOLLOWER
I20260811 00:16:45.075395 4069 heartbeater.cc:503] Master 127.7.94.62:40801 was elected leader, sending a full tablet report...
I20260811 00:16:45.075932 4101 ts_tablet_manager.cc:1428] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260811 00:16:45.075881 4098 consensus_queue.cc:260] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 } } peers { permanent_uuid: "b104e0a1d2034c489e89a15447205e59" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35975 } } peers { permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 } }
I20260811 00:16:45.077011 3848 heartbeater.cc:503] Master 127.7.94.62:40801 was elected leader, sending a full tablet report...
I20260811 00:16:45.077775 4098 ts_tablet_manager.cc:1428] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
W20260811 00:16:45.174443 4070 tablet.cc:2369] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:16:45.180780 3850 tablet.cc:2369] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:16:45.199816 4106 raft_consensus.cc:482] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:16:45.200132 4106 raft_consensus.cc:504] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 } } peers { permanent_uuid: "b104e0a1d2034c489e89a15447205e59" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35975 } } peers { permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 } }
I20260811 00:16:45.202148 4106 leader_election.cc:288] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f39c6323ef42a8842b3d870b90ce2e (127.7.94.1:42897), 2b77d2e865944dc0b522cf39c4aeeb92 (127.7.94.3:42797)
W20260811 00:16:45.202484 3629 tablet.cc:2369] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:16:45.208508 4088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.202560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:44147 (local address 127.7.94.1:42897)
0811 00:16:45.202810 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:45.202819 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:45.203412 (+ 593us) server_negotiation.cc:407] Connection header received
0811 00:16:45.203639 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:45.203643 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:45.203701 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:45.203795 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:16:45.204341 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.205030 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.206814 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.207554 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.207715 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:45.207830 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:45.208130 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 00:16:45.208259 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 00:16:45.208595 4108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.202510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:42897 (local address 127.7.94.2:44147)
0811 00:16:45.203302 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:45.203317 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 00:16:45.203482 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:16:45.203819 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:16:45.203824 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:16:45.203865 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:16:45.204192 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:45.204201 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.205172 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.205179 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:45.206648 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:45.206661 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.207777 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.207783 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:45.207961 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:45.207984 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 00:16:45.208093 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 00:16:45.208231 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":487,"threads_started":1}
I20260811 00:16:45.208949 4092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.202884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:45167 (local address 127.7.94.3:42797)
0811 00:16:45.203072 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:45.203077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:45.203930 (+ 853us) server_negotiation.cc:407] Connection header received
0811 00:16:45.204159 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:45.204164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:45.204232 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:45.204326 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:16:45.204842 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.205593 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.207237 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.208022 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.208178 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:45.208285 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:45.208565 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 00:16:45.208702 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 00:16:45.209055 4111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.202510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:42797 (local address 127.7.94.2:45167)
0811 00:16:45.203830 (+ 1320us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:45.203843 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 00:16:45.204009 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:16:45.204344 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:16:45.204348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:16:45.204372 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:16:45.204696 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:45.204705 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.205725 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.205732 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:45.207074 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:16:45.207085 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.208243 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.208250 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:16:45.208411 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:45.208434 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 00:16:45.208538 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 00:16:45.208670 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1205,"mutex_wait_us":686,"thread_start_us":319,"threads_started":1}
I20260811 00:16:45.209657 3545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47ba39d799314e7cb94e7ac7acd19cd3" candidate_uuid: "b104e0a1d2034c489e89a15447205e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f39c6323ef42a8842b3d870b90ce2e" is_pre_election: true
I20260811 00:16:45.210281 3545 raft_consensus.cc:2454] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b104e0a1d2034c489e89a15447205e59 in term 0.
I20260811 00:16:45.210175 3993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47ba39d799314e7cb94e7ac7acd19cd3" candidate_uuid: "b104e0a1d2034c489e89a15447205e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" is_pre_election: true
I20260811 00:16:45.210773 3993 raft_consensus.cc:2454] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b104e0a1d2034c489e89a15447205e59 in term 0.
I20260811 00:16:45.211017 3658 leader_election.cc:302] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f39c6323ef42a8842b3d870b90ce2e, b104e0a1d2034c489e89a15447205e59; no voters:
I20260811 00:16:45.211483 4106 raft_consensus.cc:2781] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:16:45.211647 4106 raft_consensus.cc:482] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:16:45.211796 4106 raft_consensus.cc:3037] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:16:45.214088 4106 raft_consensus.cc:504] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 } } peers { permanent_uuid: "b104e0a1d2034c489e89a15447205e59" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35975 } } peers { permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 } }
I20260811 00:16:45.214702 4106 leader_election.cc:288] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [CANDIDATE]: Term 1 election: Requested vote from peers 64f39c6323ef42a8842b3d870b90ce2e (127.7.94.1:42897), 2b77d2e865944dc0b522cf39c4aeeb92 (127.7.94.3:42797)
I20260811 00:16:45.215222 3993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47ba39d799314e7cb94e7ac7acd19cd3" candidate_uuid: "b104e0a1d2034c489e89a15447205e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b77d2e865944dc0b522cf39c4aeeb92"
I20260811 00:16:45.215242 3545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47ba39d799314e7cb94e7ac7acd19cd3" candidate_uuid: "b104e0a1d2034c489e89a15447205e59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f39c6323ef42a8842b3d870b90ce2e"
I20260811 00:16:45.215415 3993 raft_consensus.cc:3037] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:16:45.215445 3545 raft_consensus.cc:3037] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:16:45.217762 3545 raft_consensus.cc:2454] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b104e0a1d2034c489e89a15447205e59 in term 1.
I20260811 00:16:45.217818 3993 raft_consensus.cc:2454] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b104e0a1d2034c489e89a15447205e59 in term 1.
I20260811 00:16:45.218211 3658 leader_election.cc:302] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f39c6323ef42a8842b3d870b90ce2e, b104e0a1d2034c489e89a15447205e59; no voters:
I20260811 00:16:45.218564 4106 raft_consensus.cc:2781] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:16:45.218976 4106 raft_consensus.cc:686] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [term 1 LEADER]: Becoming Leader. State: Replica: b104e0a1d2034c489e89a15447205e59, State: Running, Role: LEADER
I20260811 00:16:45.219452 4106 consensus_queue.cc:237] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 } } peers { permanent_uuid: "b104e0a1d2034c489e89a15447205e59" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35975 } } peers { permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 } }
I20260811 00:16:45.224464 3304 catalog_manager.cc:5495] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 reported cstate change: term changed from 0 to 1, leader changed from <none> to b104e0a1d2034c489e89a15447205e59 (127.7.94.2). New cstate: current_term: 1 leader_uuid: "b104e0a1d2034c489e89a15447205e59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b104e0a1d2034c489e89a15447205e59" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 } health_report { overall_health: UNKNOWN } } }
I20260811 00:16:45.256732 4089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.251118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60674 (local address 127.7.94.2:35975)
0811 00:16:45.251341 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:45.251349 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:45.251368 (+ 19us) server_negotiation.cc:407] Connection header received
0811 00:16:45.251635 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:45.251639 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:45.251741 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:45.251895 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:16:45.252315 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.253051 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.254523 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.254954 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.255114 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:45.255631 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:16:45.256125 (+ 494us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:16:45.256256 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:45.256353 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 00:16:45.256422 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 00:16:45.265293 3993 raft_consensus.cc:1261] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92 [term 1 FOLLOWER]: Refusing update from remote peer b104e0a1d2034c489e89a15447205e59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:16:45.265288 3545 raft_consensus.cc:1261] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e [term 1 FOLLOWER]: Refusing update from remote peer b104e0a1d2034c489e89a15447205e59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:16:45.266299 4106 consensus_queue.cc:1035] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b77d2e865944dc0b522cf39c4aeeb92" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 42797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:16:45.266602 4112 consensus_queue.cc:1035] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f39c6323ef42a8842b3d870b90ce2e" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:16:45.282572 4117 mvcc.cc:205] Tried to move back new op lower bound from 7317124731955929088 to 7317124731782787072. Current Snapshot: MvccSnapshot[applied={T|T < 7317124731955929088}]
I20260811 00:16:45.294260 3301 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:16:45.294607 3301 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:16:45.298529 3301 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22: Sending DeleteTablet for 3 replicas of tablet 47ba39d799314e7cb94e7ac7acd19cd3
I20260811 00:16:45.299818 3513 tablet_service.cc:1514] Processing DeleteTablet for tablet 47ba39d799314e7cb94e7ac7acd19cd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:16:45 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:44850
I20260811 00:16:45.299829 3735 tablet_service.cc:1514] Processing DeleteTablet for tablet 47ba39d799314e7cb94e7ac7acd19cd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:16:45 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:60672
I20260811 00:16:45.300233 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 3368
I20260811 00:16:45.301523 3967 tablet_service.cc:1514] Processing DeleteTablet for tablet 47ba39d799314e7cb94e7ac7acd19cd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:16:45 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:52258
I20260811 00:16:45.302465 4132 tablet_replica.cc:330] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92: stopping tablet replica
I20260811 00:16:45.302912 4132 raft_consensus.cc:2229] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:16:45.303449 4132 raft_consensus.cc:2258] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:16:45.304210 4130 tablet_replica.cc:330] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e: stopping tablet replica
I20260811 00:16:45.304751 4130 raft_consensus.cc:2229] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:16:45.307174 4132 ts_tablet_manager.cc:1905] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:16:45.307739 4130 raft_consensus.cc:2258] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:16:45.313532 4131 tablet_replica.cc:330] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59: stopping tablet replica
I20260811 00:16:45.314198 4131 raft_consensus.cc:2229] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:16:45.315033 4131 raft_consensus.cc:2258] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:16:45.315834 4130 ts_tablet_manager.cc:1905] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:16:45.317413 4131 ts_tablet_manager.cc:1905] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:16:45.320924 4088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.301275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44866 (local address 127.7.94.1:42897)
0811 00:16:45.301450 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:45.301456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:45.301490 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:16:45.301565 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:45.301570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:45.301680 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:45.301781 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:45.304049 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.305301 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.307696 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.308429 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.308565 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:45.313072 (+ 4507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:45.313088 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:45.313107 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:45.313148 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:45.317303 (+ 4155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:45.318445 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:45.318454 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:45.318458 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:45.318676 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:45.319109 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:45.319114 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:45.319117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:45.319589 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:45.319956 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:45.320254 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 00:16:45.320503 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 00:16:45.321730 3620 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:16:45.323288 4132 ts_tablet_manager.cc:1918] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:16:45.323504 4132 log.cc:1196] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/47ba39d799314e7cb94e7ac7acd19cd3
I20260811 00:16:45.324183 4132 ts_tablet_manager.cc:1939] T 47ba39d799314e7cb94e7ac7acd19cd3 P 2b77d2e865944dc0b522cf39c4aeeb92: Deleting consensus metadata
I20260811 00:16:45.324800 4131 ts_tablet_manager.cc:1918] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:16:45.324976 4131 log.cc:1196] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/47ba39d799314e7cb94e7ac7acd19cd3
I20260811 00:16:45.325630 4131 ts_tablet_manager.cc:1939] T 47ba39d799314e7cb94e7ac7acd19cd3 P b104e0a1d2034c489e89a15447205e59: Deleting consensus metadata
I20260811 00:16:45.326572 3283 catalog_manager.cc:4841] TS 2b77d2e865944dc0b522cf39c4aeeb92 (127.7.94.3:42797): tablet 47ba39d799314e7cb94e7ac7acd19cd3 (table test-table [id=52f1ca93cfa04456812658887ab740b9]) successfully deleted
I20260811 00:16:45.328081 3283 catalog_manager.cc:4841] TS b104e0a1d2034c489e89a15447205e59 (127.7.94.2:35975): tablet 47ba39d799314e7cb94e7ac7acd19cd3 (table test-table [id=52f1ca93cfa04456812658887ab740b9]) successfully deleted
I20260811 00:16:45.438813 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 3368
I20260811 00:16:45.472400 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 3636
W20260811 00:16:45.472513 3282 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.94.1:42897 (error 108)
I20260811 00:16:45.474959 4091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.474421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:42897 (local address 127.0.0.1:44874)
0811 00:16:45.474646 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:45.474769 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260811 00:16:45.475378 3282 catalog_manager.cc:4584] TS 64f39c6323ef42a8842b3d870b90ce2e (127.7.94.1:42897): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 47ba39d799314e7cb94e7ac7acd19cd3: Network error: Client connection negotiation failed: client connection to 127.7.94.1:42897: connect: Connection refused (error 111)
I20260811 00:16:45.489823 4089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.474066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60678 (local address 127.7.94.2:35975)
0811 00:16:45.474273 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:45.474282 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:45.474306 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:16:45.474397 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:45.474404 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:45.474494 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:45.474621 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:45.476581 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.477610 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.479091 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.479591 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.479754 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:45.482907 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:45.482924 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:45.482951 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:45.482998 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:45.486479 (+ 3481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:45.487345 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:45.487354 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:45.487356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:45.487519 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:45.487976 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:45.487989 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:45.487991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:45.488406 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:45.488824 (+ 418us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:45.489143 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 00:16:45.489419 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 00:16:45.490645 3839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:16:45.529062 4091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.528611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:42897 (local address 127.0.0.1:44888)
0811 00:16:45.528821 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:45.528921 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 00:16:45.530197 4091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.529866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:42897 (local address 127.0.0.1:44896)
0811 00:16:45.530007 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:45.530077 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 00:16:45.590946 4091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.590433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:42897 (local address 127.0.0.1:44906)
0811 00:16:45.590665 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:45.590768 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 00:16:45.592072 4091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.591735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:42897 (local address 127.0.0.1:44920)
0811 00:16:45.591896 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:45.591963 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 00:16:45.609436 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 3636
I20260811 00:16:45.639264 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 3859
I20260811 00:16:45.653712 4092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.639981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52264 (local address 127.7.94.3:42797)
0811 00:16:45.640157 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 00:16:45.640166 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:16:45.640231 (+ 65us) server_negotiation.cc:407] Connection header received
0811 00:16:45.640455 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:16:45.640460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:16:45.640544 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:16:45.640653 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:16:45.642348 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.643071 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.644363 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:16:45.644758 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:16:45.644892 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:16:45.647738 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:16:45.647765 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:16:45.647781 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:16:45.647831 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:16:45.650627 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:45.651332 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:45.651342 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:45.651347 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:45.651495 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:16:45.651887 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:16:45.651892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:16:45.651896 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:16:45.652268 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:16:45.652590 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 00:16:45.652928 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 00:16:45.653370 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 00:16:45.654383 4059 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:16:45.673070 4091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.672493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:42897 (local address 127.0.0.1:44934)
0811 00:16:45.672737 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:45.672853 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 00:16:45.674340 4091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.673863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:42897 (local address 127.0.0.1:44940)
0811 00:16:45.674079 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:45.674194 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260811 00:16:45.767011 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 3859
I20260811 00:16:45.790091 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 3257
I20260811 00:16:45.791365 3358 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:16:45.907563 4091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:16:45.906717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:42897 (local address 127.0.0.1:44952)
0811 00:16:45.907186 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 00:16:45.907324 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252}
I20260811 00:16:45.907627 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 3257
I20260811 00:17:05.938724 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:40801
--webserver_interface=127.7.94.62
--webserver_port=46187
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:40801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:17:06.287580 4138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:06.287861 4138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:06.287968 4138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:06.295293 4138 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 00:17:06.295392 4138 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 00:17:06.295473 4138 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 00:17:06.295574 4138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:17:06.295660 4138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:06.295784 4138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:06.295863 4138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:17:06.295944 4138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:17:06.306047 4138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:40801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:40801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=46187
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:06.307391 4138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:06.308799 4138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:06.320969 4146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:06.321008 4149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:06.320938 4147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:06.321715 4138 server_base.cc:1034] running on GCE node
I20260811 00:17:06.322526 4138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:06.323933 4138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:06.325179 4138 hybrid_clock.cc:648] HybridClock initialized: now 1786407426325104 us; error 49 us; skew 500 ppm
I20260811 00:17:06.325564 4138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:06 dist-test-slave-8hhm krb5kdc[2318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407426, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:06.328738 4138 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:17:06.331960 4138 webserver.cc:460] Webserver started at http://127.7.94.62:46187/ using document root <none> and password file <none>
I20260811 00:17:06.332671 4138 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:06.332803 4138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:06.338361 4138 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:17:06.341161 4167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:06.342480 4138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:17:06.342633 4138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "4d5f62bf2ddd4264b716ec18b3554d22"
format_stamp: "Formatted at 2026-08-11 00:16:20 on dist-test-slave-8hhm"
server_key: "9c3059d80090978456b5302472bd99c1"
server_key_iv: "6c5f2409687018dc568f8e554105130a"
server_key_version: "encryptionkey@0"
I20260811 00:17:06.343575 4138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:06.356459 4138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:06.360790 4138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:06.361019 4138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:06.379631 4138 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:40801
I20260811 00:17:06.379715 4251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:40801 every 8 connection(s)
I20260811 00:17:06.381263 4138 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:17:06.387989 4254 sys_catalog.cc:263] Verifying existing consensus state
I20260811 00:17:06.390259 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 4138
I20260811 00:17:06.391374 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:42897
--local_ip_for_outbound_sockets=127.7.94.1
--tserver_master_addrs=127.7.94.62:40801
--webserver_port=35333
--webserver_interface=127.7.94.1
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:17:06.392007 4254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22: Bootstrap starting.
I20260811 00:17:06.417095 4254 log.cc:824] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:06.433177 4254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 00:17:06.433780 4254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22: Bootstrap complete.
I20260811 00:17:06.443118 4254 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40801 } }
I20260811 00:17:06.444051 4254 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d5f62bf2ddd4264b716ec18b3554d22, State: Initialized, Role: FOLLOWER
I20260811 00:17:06.444777 4254 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40801 } }
I20260811 00:17:06.445008 4254 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:17:06.445152 4254 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:17:06.445389 4254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 2 FOLLOWER]: Advancing to term 3
I20260811 00:17:06.449540 4254 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40801 } }
I20260811 00:17:06.450037 4254 leader_election.cc:302] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d5f62bf2ddd4264b716ec18b3554d22; no voters:
I20260811 00:17:06.450726 4254 leader_election.cc:288] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 00:17:06.450860 4264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 00:17:06.453190 4254 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:17:06.453523 4264 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [term 3 LEADER]: Becoming Leader. State: Replica: 4d5f62bf2ddd4264b716ec18b3554d22, State: Running, Role: LEADER
I20260811 00:17:06.454138 4264 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40801 } }
I20260811 00:17:06.457008 4265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40801 } } }
I20260811 00:17:06.457634 4265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:06.457067 4267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d5f62bf2ddd4264b716ec18b3554d22. Latest consensus state: current_term: 3 leader_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d5f62bf2ddd4264b716ec18b3554d22" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40801 } } }
I20260811 00:17:06.457782 4267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22 [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:06.462811 4276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:17:06.473357 4276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:17:06.475337 4276 catalog_manager.cc:1259] Loaded cluster ID: 5eaf0bddeab44da78a5fcacb99d6b83f
I20260811 00:17:06.475509 4276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:17:06.481035 4276 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:17:06.483135 4276 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22: Loaded TSK: 0
I20260811 00:17:06.483352 4276 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4d5f62bf2ddd4264b716ec18b3554d22: Loaded TSK: 1
F20260811 00:17:06.484735 4276 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f97c4e979ad google::LogMessage::Fail() at ??:0
@ 0x7f97c4e9a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f97c4e974de google::LogMessage::Flush() at ??:0
@ 0x7f97c4e98a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f97c8bfb7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f97c8bea73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f97da8ca1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f97da8c9783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f97da9552a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f97da9550c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f97da99f5e9 std::function<>::operator()() at ??:0
@ 0x7f97da8cdcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f97da8cbc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f97da953610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f97da953439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f97c7ae3345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f97c7ae691c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f97c7ae6799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f97c7ac1eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f97cad9b6db start_thread at ??:0
@ 0x7f97c3d3161f clone at ??:0
W20260811 00:17:06.810727 4261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:06.811053 4261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:06.811216 4261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:06.818552 4261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:06.818670 4261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:06.818848 4261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:17:06.828732 4261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:42897
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=35333
--tserver_master_addrs=127.7.94.62:40801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:06.830150 4261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:06.831502 4261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:06.844802 4343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:06.844758 4346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:06.844842 4344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:06.845657 4261 server_base.cc:1034] running on GCE node
I20260811 00:17:06.846231 4261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:06.847093 4261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:06.848311 4261 hybrid_clock.cc:648] HybridClock initialized: now 1786407426848259 us; error 44 us; skew 500 ppm
I20260811 00:17:06.848661 4261 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:06 dist-test-slave-8hhm krb5kdc[2318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407426, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:06.851861 4261 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:17:06.853493 4261 webserver.cc:460] Webserver started at http://127.7.94.1:35333/ using document root <none> and password file <none>
I20260811 00:17:06.854198 4261 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:06.854342 4261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:06.859722 4261 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 00:17:06.862701 4355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:06.863960 4261 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 00:17:06.864166 4261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "64f39c6323ef42a8842b3d870b90ce2e"
format_stamp: "Formatted at 2026-08-11 00:16:20 on dist-test-slave-8hhm"
server_key: "b21b4bf7975842607252b11e3eeb3642"
server_key_iv: "8c91d7630c72243ae736ed371d510efd"
server_key_version: "encryptionkey@0"
I20260811 00:17:06.865145 4261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:06.877563 4261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:06.882644 4261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:06.882998 4261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:06.884162 4261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:06.886854 4368 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260811 00:17:06.893586 4368 ts_tablet_manager.cc:1852] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260811 00:17:06.894582 4368 ts_tablet_manager.cc:1905] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:17:06.901935 4368 ts_tablet_manager.cc:1918] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:17:06.902105 4368 log.cc:1196] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/47ba39d799314e7cb94e7ac7acd19cd3
I20260811 00:17:06.902521 4368 ts_tablet_manager.cc:1939] T 47ba39d799314e7cb94e7ac7acd19cd3 P 64f39c6323ef42a8842b3d870b90ce2e: Deleting consensus metadata
I20260811 00:17:06.902863 4261 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260811 00:17:06.903004 4261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.017s user 0.001s sys 0.000s
I20260811 00:17:06.903163 4261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:06.903251 4261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:06.938380 4261 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:42897
I20260811 00:17:06.938467 4573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:42897 every 8 connection(s)
I20260811 00:17:06.939805 4261 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:17:06.941318 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 4261
I20260811 00:17:06.942132 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:35975
--local_ip_for_outbound_sockets=127.7.94.2
--tserver_master_addrs=127.7.94.62:40801
--webserver_port=43887
--webserver_interface=127.7.94.2
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:17:06.945648 4578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:06.943260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.1:46043)
0811 00:17:06.944179 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:06.944394 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":323,"threads_started":1}
W20260811 00:17:06.946400 4574 heartbeater.cc:644] Failed to heartbeat to 127.7.94.62:40801 (0 consecutive failures): Network error: Failed to ping master at 127.7.94.62:40801: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
I20260811 00:17:06.947369 4578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:06.946920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.1:36171)
0811 00:17:06.947122 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:06.947214 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 00:17:06.948338 4578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:06.947914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.1:38585)
0811 00:17:06.948078 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:06.948174 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 00:17:06.948616 4574 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 00:17:07.287046 4579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:07.287387 4579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:07.287547 4579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:07.295156 4579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:07.295305 4579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:07.295469 4579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:17:07.305335 4579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:35975
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=43887
--tserver_master_addrs=127.7.94.62:40801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:07.306691 4579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:07.308033 4579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:07.321506 4587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:07.321408 4592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:07.321527 4590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:07.322772 4579 server_base.cc:1034] running on GCE node
I20260811 00:17:07.323449 4579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:07.324393 4579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:07.325589 4579 hybrid_clock.cc:648] HybridClock initialized: now 1786407427325549 us; error 36 us; skew 500 ppm
I20260811 00:17:07.325904 4579 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:07 dist-test-slave-8hhm krb5kdc[2318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407427, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:07.328994 4579 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:17:07.330557 4579 webserver.cc:460] Webserver started at http://127.7.94.2:43887/ using document root <none> and password file <none>
I20260811 00:17:07.331239 4579 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:07.331378 4579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:07.336755 4579 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:17:07.339552 4608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:07.340868 4579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 00:17:07.341048 4579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "b104e0a1d2034c489e89a15447205e59"
format_stamp: "Formatted at 2026-08-11 00:16:21 on dist-test-slave-8hhm"
server_key: "51f96b77b91813a53e5180888c06ebb8"
server_key_iv: "c3d653e05b04f6e31eb669ff31578dec"
server_key_version: "encryptionkey@0"
I20260811 00:17:07.342090 4579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:07.365540 4579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:07.369881 4579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:07.370177 4579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:07.371218 4579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:07.372684 4579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:17:07.372821 4579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:07.372943 4579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:07.373029 4579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:07.409307 4579 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:35975
I20260811 00:17:07.409399 4808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:35975 every 8 connection(s)
I20260811 00:17:07.410859 4579 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:17:07.413506 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 4579
I20260811 00:17:07.414331 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:42797
--local_ip_for_outbound_sockets=127.7.94.3
--tserver_master_addrs=127.7.94.62:40801
--webserver_port=45553
--webserver_interface=127.7.94.3
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:17:07.416543 4812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:07.414415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.2:32963)
0811 00:17:07.415270 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:07.415476 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":345,"threads_started":1}
W20260811 00:17:07.417416 4810 heartbeater.cc:644] Failed to heartbeat to 127.7.94.62:40801 (0 consecutive failures): Network error: Failed to ping master at 127.7.94.62:40801: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
I20260811 00:17:07.418329 4812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:07.417963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.2:38029)
0811 00:17:07.418132 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:07.418216 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 00:17:07.419711 4812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:07.419073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.2:51005)
0811 00:17:07.419424 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:07.419537 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
W20260811 00:17:07.420064 4810 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 00:17:07.760874 4815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:07.761160 4815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:07.761332 4815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:07.768422 4815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:07.768545 4815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:07.768786 4815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:17:07.779389 4815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:42797
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=45553
--tserver_master_addrs=127.7.94.62:40801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:07.780821 4815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:07.782186 4815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:07.795722 4831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:07.795694 4827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:07.795761 4826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:07.796597 4815 server_base.cc:1034] running on GCE node
I20260811 00:17:07.797343 4815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:07.798312 4815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:07.799590 4815 hybrid_clock.cc:648] HybridClock initialized: now 1786407427799522 us; error 60 us; skew 500 ppm
I20260811 00:17:07.799908 4815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:07 dist-test-slave-8hhm krb5kdc[2318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407427, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:07.803030 4815 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:17:07.804508 4815 webserver.cc:460] Webserver started at http://127.7.94.3:45553/ using document root <none> and password file <none>
I20260811 00:17:07.805189 4815 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:07.805400 4815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:07.810860 4815 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:17:07.815263 4842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:07.816571 4815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:17:07.816759 4815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "2b77d2e865944dc0b522cf39c4aeeb92"
format_stamp: "Formatted at 2026-08-11 00:16:21 on dist-test-slave-8hhm"
server_key: "b9f16ee39b6288354622e48e5f3fb461"
server_key_iv: "f20c3d7ec890349495f7699d9f6fdfbd"
server_key_version: "encryptionkey@0"
I20260811 00:17:07.817798 4815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:07.844111 4815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:07.848269 4815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:07.848556 4815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:07.849617 4815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:07.850988 4815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:17:07.851120 4815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:07.851257 4815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:07.851347 4815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:07.885061 4815 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:42797
I20260811 00:17:07.885124 5046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:42797 every 8 connection(s)
I20260811 00:17:07.886605 4815 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:17:07.887616 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 4815
I20260811 00:17:07.891604 5051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:07.889821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.3:56173)
0811 00:17:07.890526 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:07.890739 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":284,"threads_started":1}
W20260811 00:17:07.892238 5047 heartbeater.cc:644] Failed to heartbeat to 127.7.94.62:40801 (0 consecutive failures): Network error: Failed to ping master at 127.7.94.62:40801: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
I20260811 00:17:07.892992 5051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:07.892708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.3:57379)
0811 00:17:07.892830 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:07.892895 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 00:17:07.893823 5051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:07.893529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.3:46529)
0811 00:17:07.893649 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:07.893719 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 00:17:07.894078 5047 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 00:17:07.949993 5053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:07.949160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.1:50483)
0811 00:17:07.949675 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:07.949759 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":278,"threads_started":1}
I20260811 00:17:08.421456 5054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:08.420554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.2:38365)
0811 00:17:08.420988 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:08.421070 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":219,"threads_started":1}
I20260811 00:17:08.895285 5055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:08.894546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.3:45685)
0811 00:17:08.894976 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:08.895059 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":207,"threads_started":1}
I20260811 00:17:08.951452 5056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:08.950800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.1:39389)
0811 00:17:08.951220 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:08.951308 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":219,"threads_started":1}
I20260811 00:17:09.423130 5057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:09.422378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.2:37655)
0811 00:17:09.422855 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:09.422937 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":220,"threads_started":1}
I20260811 00:17:09.897024 5059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:09.896299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.3:36845)
0811 00:17:09.896785 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:09.896862 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":249,"threads_started":1}
I20260811 00:17:09.952991 5063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:09.952296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.1:50163)
0811 00:17:09.952735 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:09.952817 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":233,"threads_started":1}
I20260811 00:17:10.424633 5064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:10.423926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.2:44025)
0811 00:17:10.424354 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:10.424431 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":219,"threads_started":1}
I20260811 00:17:10.898608 5065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:10.897903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.3:35571)
0811 00:17:10.898360 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:10.898445 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":255,"threads_started":1}
I20260811 00:17:10.954785 5067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:10.953927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.1:36241)
0811 00:17:10.954503 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:10.954585 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":325,"threads_started":1}
I20260811 00:17:11.426294 5068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:11.425557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.2:46519)
0811 00:17:11.426017 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:11.426111 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":237,"threads_started":1}
I20260811 00:17:11.900295 5069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:11.899511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.3:40937)
0811 00:17:11.900008 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:11.900102 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":257,"threads_started":1}
I20260811 00:17:11.956317 5071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:11.955637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.1:40637)
0811 00:17:11.956082 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:11.956157 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":212,"threads_started":1}
I20260811 00:17:12.428053 5072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:12.427203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.2:52959)
0811 00:17:12.427793 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:12.427880 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":328,"threads_started":1}
I20260811 00:17:12.901963 5073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:12.901264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.3:34741)
0811 00:17:12.901729 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:12.901804 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":223,"threads_started":1}
I20260811 00:17:12.958109 5074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:12.957387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.1:47621)
0811 00:17:12.957872 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:12.957951 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":258,"threads_started":1}
I20260811 00:17:13.429772 5078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:13.429009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.2:51131)
0811 00:17:13.429479 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:13.429591 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":272,"threads_started":1}
I20260811 00:17:13.903534 5080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:13.902812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.3:58305)
0811 00:17:13.903264 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:13.903374 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":253,"threads_started":1}
I20260811 00:17:13.959836 5081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:13.959008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.1:33253)
0811 00:17:13.959552 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:13.959634 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":273,"threads_started":1}
I20260811 00:17:14.431517 5082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:14.430715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.2:33425)
0811 00:17:14.431221 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:14.431302 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":236,"threads_started":1}
I20260811 00:17:14.905407 5089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:14.904501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.3:60319)
0811 00:17:14.905023 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:14.905114 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":236,"threads_started":1}
I20260811 00:17:14.961723 5090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:14.960897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.1:44153)
0811 00:17:14.961436 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:14.961542 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":250,"threads_started":1}
I20260811 00:17:15.433391 5092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:15.432531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.2:55617)
0811 00:17:15.432998 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:15.433084 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":253,"threads_started":1}
I20260811 00:17:15.908211 5094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:15.906304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.3:54449)
0811 00:17:15.907956 (+ 1652us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:15.908037 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1552,"thread_start_us":1441,"threads_started":1}
I20260811 00:17:15.963449 5098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:15.962652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.1:47657)
0811 00:17:15.963129 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:15.963279 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":232,"threads_started":1}
I20260811 00:17:16.435279 5099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:16.434429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.2:50011)
0811 00:17:16.434987 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:16.435072 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":311,"threads_started":1}
I20260811 00:17:16.909858 5102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:16.909077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.3:41805)
0811 00:17:16.909588 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:16.909673 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":281,"threads_started":1}
I20260811 00:17:16.965101 5103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:16.964444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.1:40589)
0811 00:17:16.964864 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:16.964941 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":224,"threads_started":1}
I20260811 00:17:17.436993 5107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:17.436313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.2:38659)
0811 00:17:17.436748 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:17.436826 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":237,"threads_started":1}
I20260811 00:17:17.911585 5108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:17.910874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.3:60841)
0811 00:17:17.911340 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:17.911420 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":269,"threads_started":1}
I20260811 00:17:17.966801 5110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:17.965988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.1:43599)
0811 00:17:17.966518 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:17.966617 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":299,"threads_started":1}
I20260811 00:17:18.438777 5112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:18.438051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.2:36269)
0811 00:17:18.438503 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:18.438587 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":212,"threads_started":1}
I20260811 00:17:18.913569 5113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:18.912735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.3:52823)
0811 00:17:18.913281 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:18.913383 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":257,"threads_started":1}
I20260811 00:17:18.968619 5114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:18.967759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.1:48797)
0811 00:17:18.968380 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:18.968461 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":386,"threads_started":1}
I20260811 00:17:19.440454 5115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:19.439749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.2:40973)
0811 00:17:19.440198 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:19.440292 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":233,"threads_started":1}
I20260811 00:17:19.915373 5116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:19.914585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.3:36765)
0811 00:17:19.915103 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:19.915216 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":286,"threads_started":1}
I20260811 00:17:19.970443 5118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:19.969547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.1:38223)
0811 00:17:19.970062 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:19.970144 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":234,"threads_started":1}
I20260811 00:17:20.442307 5121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:20.441488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.2:38057)
0811 00:17:20.442023 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:20.442113 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":262,"threads_started":1}
I20260811 00:17:20.917012 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:20.916293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.3:51585)
0811 00:17:20.916736 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:20.916822 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":213,"threads_started":1}
I20260811 00:17:20.972121 5124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:20.971383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.1:42023)
0811 00:17:20.971853 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:20.971936 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":226,"threads_started":1}
I20260811 00:17:21.444022 5125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:21.443236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.2:40811)
0811 00:17:21.443746 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:21.443835 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":272,"threads_started":1}
I20260811 00:17:21.918716 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:21.918024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.3:45945)
0811 00:17:21.918455 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:21.918536 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":223,"threads_started":1}
I20260811 00:17:21.974049 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:21.973324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.1:57105)
0811 00:17:21.973776 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:21.973856 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":220,"threads_started":1}
I20260811 00:17:22.446143 5130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:22.445357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.2:59893)
0811 00:17:22.445808 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:22.445898 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":232,"threads_started":1}
I20260811 00:17:22.899407 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 4261
I20260811 00:17:22.920667 5132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:22.919854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40801 (local address 127.7.94.3:33081)
0811 00:17:22.920361 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:22.920445 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.62:40801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":285,"threads_started":1}
I20260811 00:17:22.921562 5131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:22.900664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38882 (local address 127.7.94.1:42897)
0811 00:17:22.901295 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:22.901309 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:22.901348 (+ 39us) server_negotiation.cc:407] Connection header received
0811 00:17:22.902976 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:22.902986 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:22.903406 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:22.903715 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:22.905045 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:22.906362 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:22.907635 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:22.908259 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:22.908383 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:22.915310 (+ 6927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:22.915318 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:22.915334 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:22.915370 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:22.918188 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:22.918775 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:22.918781 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:22.918787 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:22.918941 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:22.919314 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:22.919318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:22.919321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:22.919929 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:22.920247 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:22.920698 (+ 451us) server_negotiation.cc:299] Negotiation successful
0811 00:17:22.921160 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":306,"threads_started":1}
I20260811 00:17:22.922940 4565 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:23.010828 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 4261
I20260811 00:17:23.040776 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 4579
I20260811 00:17:23.058115 5136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:23.041798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41210 (local address 127.7.94.2:35975)
0811 00:17:23.042310 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:23.042324 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:23.042361 (+ 37us) server_negotiation.cc:407] Connection header received
0811 00:17:23.043888 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:23.043898 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:23.044337 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:23.044634 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:23.046183 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:23.047577 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:23.048783 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:23.049493 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:23.049620 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:23.052270 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:23.052277 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:23.052291 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:23.052322 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:23.055180 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:23.055917 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:23.055923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:23.055928 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:23.056091 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:23.056432 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:23.056435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:23.056437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:23.056914 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:23.057140 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:23.057534 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 00:17:23.057837 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":236,"threads_started":1}
I20260811 00:17:23.059490 4798 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:23.141973 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 4579
I20260811 00:17:23.171312 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 4815
I20260811 00:17:23.189198 5139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:23.172193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60244 (local address 127.7.94.3:42797)
0811 00:17:23.172754 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:23.172770 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:23.172812 (+ 42us) server_negotiation.cc:407] Connection header received
0811 00:17:23.174474 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:23.174484 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:23.174911 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:23.175272 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:23.176867 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:23.178185 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:23.179387 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:23.180021 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:23.180257 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:23.182938 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:23.182946 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:23.182961 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:23.183004 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:23.185913 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:23.186589 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:23.186597 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:23.186605 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:23.186763 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:23.187169 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:23.187174 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:23.187179 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:23.187696 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:23.188041 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:23.188481 (+ 440us) server_negotiation.cc:299] Negotiation successful
0811 00:17:23.188895 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":262,"threads_started":1}
I20260811 00:17:23.190577 5039 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:23.273582 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 4815
2026-08-11T00:17:23Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65253 ms)
[----------] 30 tests from SecurityITest (309292 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:17:23 dist-test-slave-8hhm krb5kdc[5144](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:17:23 dist-test-slave-8hhm krb5kdc[5144](info): set up 2 sockets
Aug 11 00:17:23 dist-test-slave-8hhm krb5kdc[5144](info): commencing operation
krb5kdc: starting...
W20260811 00:17:24.655707 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.331s user 0.006s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:17:24 dist-test-slave-8hhm krb5kdc[5144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:17:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:17:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:17:24.964576 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:44221
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:37873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:44221
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:17:25.304251 5171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:25.304556 5171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:25.304675 5171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:25.312037 5171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:17:25.312152 5171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:25.312242 5171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:25.312354 5171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:17:25.312435 5171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:17:25.322642 5171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:37873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:44221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:44221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:25.324113 5171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:25.325564 5171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:25.338435 5183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:25.338373 5185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:25.338469 5180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:25.339485 5171 server_base.cc:1034] running on GCE node
I20260811 00:17:25.340407 5171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:25.341887 5171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:25.343089 5171 hybrid_clock.cc:648] HybridClock initialized: now 1786407445343026 us; error 51 us; skew 500 ppm
I20260811 00:17:25.343453 5171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:25 dist-test-slave-8hhm krb5kdc[5144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407445, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:25.346700 5171 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:17:25.348178 5171 webserver.cc:460] Webserver started at http://127.7.94.62:37021/ using document root <none> and password file <none>
I20260811 00:17:25.348933 5171 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:25.349092 5171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:25.349470 5171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:25.352571 5171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "bb10f164914d40d5ade7932634cddec7"
format_stamp: "Formatted at 2026-08-11 00:17:25 on dist-test-slave-8hhm"
server_key: "e2f59df1a873b0b4513e2aca243fa834"
server_key_iv: "e3ac95e91f2839a6c4038e65ae587735"
server_key_version: "encryptionkey@0"
I20260811 00:17:25.353471 5171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "bb10f164914d40d5ade7932634cddec7"
format_stamp: "Formatted at 2026-08-11 00:17:25 on dist-test-slave-8hhm"
server_key: "e2f59df1a873b0b4513e2aca243fa834"
server_key_iv: "e3ac95e91f2839a6c4038e65ae587735"
server_key_version: "encryptionkey@0"
I20260811 00:17:25.358848 5171 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 00:17:25.362401 5197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:25.363816 5171 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:17:25.363997 5171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "bb10f164914d40d5ade7932634cddec7"
format_stamp: "Formatted at 2026-08-11 00:17:25 on dist-test-slave-8hhm"
server_key: "e2f59df1a873b0b4513e2aca243fa834"
server_key_iv: "e3ac95e91f2839a6c4038e65ae587735"
server_key_version: "encryptionkey@0"
I20260811 00:17:25.364209 5171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:25.391681 5171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:25.396234 5171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:25.396536 5171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:25.415689 5171 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:44221
I20260811 00:17:25.415714 5278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:44221 every 8 connection(s)
I20260811 00:17:25.417129 5171 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:17:25.420960 5279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:25.424968 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 5171
I20260811 00:17:25.425235 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:17:25.426111 7544 external_mini_cluster.cc:1428] Setting key c8dfb7db82599a9e7b1400e00e15821e
I20260811 00:17:25.435330 5279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7: Bootstrap starting.
I20260811 00:17:25.439546 5279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:25.441045 5279 log.cc:824] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:17:25 dist-test-slave-8hhm krb5kdc[5144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:17:25.444698 5279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7: No bootstrap required, opened a new log
I20260811 00:17:25.451470 5284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:25.428412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44070 (local address 127.7.94.62:44221)
0811 00:17:25.429280 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:25.429296 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:25.429346 (+ 50us) server_negotiation.cc:407] Connection header received
0811 00:17:25.430965 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:25.430988 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:25.431407 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:25.431792 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:25.433730 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:25.435036 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:25.436986 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:25.437774 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:25.437917 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:25.443639 (+ 5722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:25.443647 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:25.443670 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:25.443705 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:25.446994 (+ 3289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:25.447856 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:25.447863 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:25.447872 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:25.448053 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:25.448463 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:25.448466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:25.448468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:25.448985 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:25.449306 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:25.449719 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 00:17:25.450124 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":377,"threads_started":1}
I20260811 00:17:25.454635 5279 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb10f164914d40d5ade7932634cddec7" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44221 } }
I20260811 00:17:25.455091 5279 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:25.455215 5279 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb10f164914d40d5ade7932634cddec7, State: Initialized, Role: FOLLOWER
I20260811 00:17:25.455796 5279 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb10f164914d40d5ade7932634cddec7" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44221 } }
I20260811 00:17:25.456044 5279 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:17:25.456194 5279 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:17:25.456379 5279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:25.458667 5279 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb10f164914d40d5ade7932634cddec7" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44221 } }
I20260811 00:17:25.459187 5279 leader_election.cc:302] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb10f164914d40d5ade7932634cddec7; no voters:
I20260811 00:17:25.459906 5279 leader_election.cc:288] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:17:25.460057 5290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:17:25.460740 5290 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7 [term 1 LEADER]: Becoming Leader. State: Replica: bb10f164914d40d5ade7932634cddec7, State: Running, Role: LEADER
I20260811 00:17:25.461319 5290 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb10f164914d40d5ade7932634cddec7" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44221 } }
I20260811 00:17:25.462337 5279 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:17:25.464644 5297 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb10f164914d40d5ade7932634cddec7. Latest consensus state: current_term: 1 leader_uuid: "bb10f164914d40d5ade7932634cddec7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb10f164914d40d5ade7932634cddec7" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44221 } } }
I20260811 00:17:25.464684 5293 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb10f164914d40d5ade7932634cddec7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb10f164914d40d5ade7932634cddec7" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 44221 } } }
I20260811 00:17:25.465026 5297 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7 [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:25.465097 5293 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7 [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:25.470275 5303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:17:25.475872 5303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:17:25.484423 5303 catalog_manager.cc:1347] Generated new cluster ID: fcd2cd51b70947c288fb101cad46ec1b
I20260811 00:17:25.484539 5303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:17:25.507215 5303 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:17:25.509107 5303 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:17:25.518571 5303 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7: Generated new TSK 0
I20260811 00:17:25.519590 5303 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
I20260811 00:17:25.597173 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:44221
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:37873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:17:25.943861 5334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:25.944157 5334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:25.944250 5334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:25.951453 5334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:25.951562 5334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:25.951776 5334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:17:25.964296 5334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:37873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:44221
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:25.966955 5334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:25.968711 5334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:25.990902 5342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:25.990938 5344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:25.991782 5348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:25.992733 5334 server_base.cc:1034] running on GCE node
I20260811 00:17:25.993701 5334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:25.996052 5334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:25.997311 5334 hybrid_clock.cc:648] HybridClock initialized: now 1786407445997216 us; error 80 us; skew 500 ppm
I20260811 00:17:25.997699 5334 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:26 dist-test-slave-8hhm krb5kdc[5144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407445, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:26.002123 5334 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:17:26.004222 5334 webserver.cc:460] Webserver started at http://127.7.94.1:37585/ using document root <none> and password file <none>
I20260811 00:17:26.005275 5334 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:26.005409 5334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:26.005776 5334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:26.010047 5334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "689a07d20b2f41a19104dedf0a51a2fd"
format_stamp: "Formatted at 2026-08-11 00:17:26 on dist-test-slave-8hhm"
server_key: "1281f30124203c38c640d335d22604c6"
server_key_iv: "1d7cb714b3e2005ec752fa984154c011"
server_key_version: "encryptionkey@0"
I20260811 00:17:26.011119 5334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "689a07d20b2f41a19104dedf0a51a2fd"
format_stamp: "Formatted at 2026-08-11 00:17:26 on dist-test-slave-8hhm"
server_key: "1281f30124203c38c640d335d22604c6"
server_key_iv: "1d7cb714b3e2005ec752fa984154c011"
server_key_version: "encryptionkey@0"
I20260811 00:17:26.016853 5334 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 00:17:26.020612 5360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:26.021927 5334 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:17:26.022097 5334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "689a07d20b2f41a19104dedf0a51a2fd"
format_stamp: "Formatted at 2026-08-11 00:17:26 on dist-test-slave-8hhm"
server_key: "1281f30124203c38c640d335d22604c6"
server_key_iv: "1d7cb714b3e2005ec752fa984154c011"
server_key_version: "encryptionkey@0"
I20260811 00:17:26.022291 5334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:26.053252 5334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:26.057844 5334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:26.058171 5334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:26.059165 5334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:26.060523 5334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:17:26.060628 5334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:26.060783 5334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:26.060856 5334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:26.095623 5334 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:42233
I20260811 00:17:26.095675 5562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:42233 every 8 connection(s)
I20260811 00:17:26.096956 5334 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:17:26.099385 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 5334
I20260811 00:17:26.099663 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:17:26.100332 7544 external_mini_cluster.cc:1428] Setting key 38abd92b0e0a1612ec6af91ff80c2eec
Aug 11 00:17:26 dist-test-slave-8hhm krb5kdc[5144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407445, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:17:26.120386 5568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:26.100650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:46853 (local address 127.7.94.62:44221)
0811 00:17:26.102793 (+ 2143us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:26.102797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:26.102813 (+ 16us) server_negotiation.cc:407] Connection header received
0811 00:17:26.103837 (+ 1024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:26.103841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:26.103903 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:26.103990 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:26.106166 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:26.106893 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:26.109346 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:26.109742 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:26.109841 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:26.114154 (+ 4313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:26.114160 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:26.114163 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:26.114193 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:26.117021 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:26.117828 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:26.117837 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:26.117843 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:26.117997 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:26.118437 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:26.118445 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:26.118451 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:26.118826 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:26.118979 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:26.119653 (+ 674us) server_negotiation.cc:299] Negotiation successful
0811 00:17:26.119919 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2049,"thread_start_us":1921,"threads_started":1}
I20260811 00:17:26.121024 5566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:26.100197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:44221 (local address 127.7.94.1:46853)
0811 00:17:26.101074 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:26.101114 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:26.103520 (+ 2406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:26.104282 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:26.104295 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:26.105121 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:17:26.105986 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:26.106015 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:26.107058 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:26.107066 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:26.109165 (+ 2099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:26.109184 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:26.109971 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:26.109982 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:26.110330 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:26.111064 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:17:26.111101 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:17:26.113911 (+ 2810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:17:26.117176 (+ 3265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:17:26.117183 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:17:26.117195 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:17:26.117691 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:17:26.118124 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:17:26.118127 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:17:26.118130 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:17:26.118319 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:17:26.118945 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:17:26.118951 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:17:26.119258 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 00:17:26.119490 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 00:17:26.119872 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":360,"threads_started":1}
I20260811 00:17:26.122687 5563 heartbeater.cc:347] Connected to a master server at 127.7.94.62:44221
I20260811 00:17:26.123025 5563 heartbeater.cc:464] Registering TS with master...
I20260811 00:17:26.123950 5563 heartbeater.cc:511] Master 127.7.94.62:44221 requested a full tablet report, sending...
I20260811 00:17:26.126288 5220 ts_manager.cc:194] Registered new tserver with Master: 689a07d20b2f41a19104dedf0a51a2fd (127.7.94.1:42233)
I20260811 00:17:26.127926 5220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:46853
WARNING: no policy specified for kudu/127.7.94.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
I20260811 00:17:26.174408 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:44221
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:37873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:17:26.516875 5574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:26.517181 5574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:26.517369 5574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:26.524600 5574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:26.524755 5574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:26.524923 5574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:17:26.535569 5574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:37873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:44221
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:26.536850 5574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:26.538244 5574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:26.551394 5586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:26.551437 5587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:26.551447 5589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:26.552378 5574 server_base.cc:1034] running on GCE node
I20260811 00:17:26.552940 5574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:26.553998 5574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:26.555197 5574 hybrid_clock.cc:648] HybridClock initialized: now 1786407446555152 us; error 33 us; skew 500 ppm
I20260811 00:17:26.555473 5574 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:26 dist-test-slave-8hhm krb5kdc[5144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407446, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:26.558576 5574 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:17:26.559896 5574 webserver.cc:460] Webserver started at http://127.7.94.2:37709/ using document root <none> and password file <none>
I20260811 00:17:26.560525 5574 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:26.560621 5574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:26.560899 5574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:26.564136 5574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "4a584b2f08d249d884a832162c4dd57f"
format_stamp: "Formatted at 2026-08-11 00:17:26 on dist-test-slave-8hhm"
server_key: "d02bd3bf86272a309b10e6dcac514f0f"
server_key_iv: "c60eab53586199902c82a6745d2bf431"
server_key_version: "encryptionkey@0"
I20260811 00:17:26.564879 5574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "4a584b2f08d249d884a832162c4dd57f"
format_stamp: "Formatted at 2026-08-11 00:17:26 on dist-test-slave-8hhm"
server_key: "d02bd3bf86272a309b10e6dcac514f0f"
server_key_iv: "c60eab53586199902c82a6745d2bf431"
server_key_version: "encryptionkey@0"
I20260811 00:17:26.569983 5574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 00:17:26.573421 5600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:26.574718 5574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:17:26.574894 5574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "4a584b2f08d249d884a832162c4dd57f"
format_stamp: "Formatted at 2026-08-11 00:17:26 on dist-test-slave-8hhm"
server_key: "d02bd3bf86272a309b10e6dcac514f0f"
server_key_iv: "c60eab53586199902c82a6745d2bf431"
server_key_version: "encryptionkey@0"
I20260811 00:17:26.575088 5574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:26.598618 5574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:26.602730 5574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:26.603037 5574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:26.603995 5574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:26.605396 5574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:17:26.605520 5574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:26.605698 5574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:26.605775 5574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:26.649681 5574 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:37519
I20260811 00:17:26.649744 5806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:37519 every 8 connection(s)
I20260811 00:17:26.650977 5574 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:17:26.655256 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 5574
I20260811 00:17:26.655453 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:17:26.656037 7544 external_mini_cluster.cc:1428] Setting key fa01f995ac0d001ab13accf6867b6525
Aug 11 00:17:26 dist-test-slave-8hhm krb5kdc[5144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407446, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:17:26.673877 5809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:26.653549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:54341 (local address 127.7.94.62:44221)
0811 00:17:26.653966 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:26.653971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:26.655106 (+ 1135us) server_negotiation.cc:407] Connection header received
0811 00:17:26.657547 (+ 2441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:26.657551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:26.657621 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:26.657714 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:26.659701 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:26.660385 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:26.662856 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:26.663401 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:26.663496 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:26.667675 (+ 4179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:26.667680 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:26.667682 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:26.667713 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:26.670585 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:26.671392 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:26.671397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:26.671400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:26.671559 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:26.672032 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:26.672038 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:26.672041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:26.672314 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:26.672486 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:26.673274 (+ 788us) server_negotiation.cc:299] Negotiation successful
0811 00:17:26.673494 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":200,"threads_started":1}
I20260811 00:17:26.674885 5812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:26.654148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:44221 (local address 127.7.94.2:54341)
0811 00:17:26.654919 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:26.654971 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:26.657263 (+ 2292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:26.657999 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:26.658007 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:26.658737 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:17:26.659525 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:26.659554 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:26.660546 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:26.660553 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:26.662715 (+ 2162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:26.662734 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:26.663626 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:26.663635 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:26.663977 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:26.664708 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:17:26.664742 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:17:26.667454 (+ 2712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:17:26.670756 (+ 3302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:17:26.670765 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:17:26.670782 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:17:26.671282 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:17:26.671676 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:17:26.671681 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:17:26.671684 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:17:26.671895 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:17:26.672467 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:17:26.672475 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:17:26.672868 (+ 393us) client_negotiation.cc:769] Sending connection context
0811 00:17:26.673126 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 00:17:26.673684 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":276,"threads_started":1}
I20260811 00:17:26.676432 5807 heartbeater.cc:347] Connected to a master server at 127.7.94.62:44221
I20260811 00:17:26.676767 5807 heartbeater.cc:464] Registering TS with master...
I20260811 00:17:26.677677 5807 heartbeater.cc:511] Master 127.7.94.62:44221 requested a full tablet report, sending...
I20260811 00:17:26.679350 5220 ts_manager.cc:194] Registered new tserver with Master: 4a584b2f08d249d884a832162c4dd57f (127.7.94.2:37519)
I20260811 00:17:26.680698 5220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:54341
WARNING: no policy specified for kudu/127.7.94.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
I20260811 00:17:26.728184 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:44221
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:37873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:17:27.075955 5817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:27.076238 5817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:27.076329 5817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:27.083418 5817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:27.083520 5817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:27.083629 5817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:17:27.096572 5817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:37873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:44221
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:27.097908 5817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:27.099340 5817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:27.113237 5827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:27.113760 5829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:27.113876 5826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:27.115177 5817 server_base.cc:1034] running on GCE node
I20260811 00:17:27.115998 5817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:27.117244 5817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:27.118503 5817 hybrid_clock.cc:648] HybridClock initialized: now 1786407447118425 us; error 56 us; skew 500 ppm
I20260811 00:17:27.118947 5817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:27 dist-test-slave-8hhm krb5kdc[5144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407447, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:27.123113 5817 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:17:27.124714 5817 webserver.cc:460] Webserver started at http://127.7.94.3:45643/ using document root <none> and password file <none>
I20260811 00:17:27.125547 5817 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:27.125694 5817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:27.126024 5817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:27.129081 5817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "351b38562b5d439184acc6e6abb2219d"
format_stamp: "Formatted at 2026-08-11 00:17:27 on dist-test-slave-8hhm"
server_key: "d0a0d11e9d3f333a6e68ebb0be259065"
server_key_iv: "24bbcb923b9c9f1cecf3acf9a37391a0"
server_key_version: "encryptionkey@0"
I20260811 00:17:27.130017 5817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "351b38562b5d439184acc6e6abb2219d"
format_stamp: "Formatted at 2026-08-11 00:17:27 on dist-test-slave-8hhm"
server_key: "d0a0d11e9d3f333a6e68ebb0be259065"
server_key_iv: "24bbcb923b9c9f1cecf3acf9a37391a0"
server_key_version: "encryptionkey@0"
I20260811 00:17:27.131757 5563 heartbeater.cc:503] Master 127.7.94.62:44221 was elected leader, sending a full tablet report...
I20260811 00:17:27.136243 5817 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260811 00:17:27.139878 5841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:27.141410 5817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:17:27.141582 5817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "351b38562b5d439184acc6e6abb2219d"
format_stamp: "Formatted at 2026-08-11 00:17:27 on dist-test-slave-8hhm"
server_key: "d0a0d11e9d3f333a6e68ebb0be259065"
server_key_iv: "24bbcb923b9c9f1cecf3acf9a37391a0"
server_key_version: "encryptionkey@0"
I20260811 00:17:27.141788 5817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:27.157806 5817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:27.163736 5817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:27.164077 5817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:27.165355 5817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:27.167235 5817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:17:27.167344 5817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:27.167484 5817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:27.167562 5817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:27.210753 5817 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:41083
I20260811 00:17:27.210772 6057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:41083 every 8 connection(s)
I20260811 00:17:27.212628 5817 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:17:27.214643 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 5817
I20260811 00:17:27.214869 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:17:27.215552 7544 external_mini_cluster.cc:1428] Setting key fa8afb34b71519104442c19a940fba4f
Aug 11 00:17:27 dist-test-slave-8hhm krb5kdc[5144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407447, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:17:27.241778 6062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.215472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:54575 (local address 127.7.94.62:44221)
0811 00:17:27.221697 (+ 6225us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:27.221706 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:27.222263 (+ 557us) server_negotiation.cc:407] Connection header received
0811 00:17:27.224105 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:27.224114 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:27.224212 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:27.224333 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:27.226305 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.227793 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.229277 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.230133 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.230282 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.234020 (+ 3738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:27.234029 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:27.234036 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:27.234082 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:27.238347 (+ 4265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:27.239049 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:27.239057 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:27.239062 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:27.239234 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:27.239653 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:27.239660 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:27.239666 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:27.240111 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:27.240277 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:27.240927 (+ 650us) server_negotiation.cc:299] Negotiation successful
0811 00:17:27.241197 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6029,"thread_start_us":5885,"threads_started":1}
I20260811 00:17:27.242293 6063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.216209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:44221 (local address 127.7.94.3:54575)
0811 00:17:27.222087 (+ 5878us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:27.222145 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:27.223845 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:27.224714 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:27.224729 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:27.225438 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:17:27.226139 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:27.226159 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.227926 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.227930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:27.229101 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:27.229113 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.230371 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.230380 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:27.230653 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.231450 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:17:27.231482 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:17:27.233792 (+ 2310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:17:27.238504 (+ 4712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:17:27.238511 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:17:27.238522 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:17:27.238914 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:17:27.239348 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:17:27.239351 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:17:27.239353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:17:27.239523 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:17:27.240246 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:17:27.240251 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:17:27.240541 (+ 290us) client_negotiation.cc:769] Sending connection context
0811 00:17:27.240755 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 00:17:27.241105 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5597,"thread_start_us":5349,"threads_started":1}
I20260811 00:17:27.243762 6059 heartbeater.cc:347] Connected to a master server at 127.7.94.62:44221
I20260811 00:17:27.244087 6059 heartbeater.cc:464] Registering TS with master...
I20260811 00:17:27.245040 6059 heartbeater.cc:511] Master 127.7.94.62:44221 requested a full tablet report, sending...
I20260811 00:17:27.246622 5220 ts_manager.cc:194] Registered new tserver with Master: 351b38562b5d439184acc6e6abb2219d (127.7.94.3:41083)
I20260811 00:17:27.247437 5220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:54575
I20260811 00:17:27.256666 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:17:27.277315 6062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.263662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44080 (local address 127.7.94.62:44221)
0811 00:17:27.263850 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:27.263862 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:27.264309 (+ 447us) server_negotiation.cc:407] Connection header received
0811 00:17:27.264524 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:27.264529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:27.264603 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:27.264713 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:27.266226 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.266964 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.268354 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.268795 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.268926 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.271548 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:27.271554 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:27.271559 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:27.271604 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:27.274518 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:27.275178 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:27.275185 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:27.275191 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:27.275322 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:27.275739 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:27.275746 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:27.275749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:27.276078 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:27.276329 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:27.276709 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 00:17:27.276938 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 00:17:27.281873 5220 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:17:27.289201 5220 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:17:27.334776 6084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.303457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:42233 (local address 127.0.0.1:38110)
0811 00:17:27.321475 (+ 18018us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:27.321500 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:27.321639 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:27.322800 (+ 1161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:27.322812 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:27.322862 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:27.323333 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:27.323349 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.325903 (+ 2554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.325911 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:27.329024 (+ 3113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:27.329044 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.333497 (+ 4453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.333505 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:27.333791 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.333871 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 00:17:27.334018 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 00:17:27.334267 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17869,"thread_start_us":17665,"threads_started":1}
I20260811 00:17:27.335985 6088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.304131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38110 (local address 127.7.94.1:42233)
0811 00:17:27.320938 (+ 16807us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:27.320950 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:27.321791 (+ 841us) server_negotiation.cc:407] Connection header received
0811 00:17:27.321914 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:27.321925 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:27.322226 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:27.322454 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:27.323557 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.325640 (+ 2083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.329300 (+ 3660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.331380 (+ 2080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.331531 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.331844 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:27.335324 (+ 3480us) server_negotiation.cc:299] Negotiation successful
0811 00:17:27.335572 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16596,"thread_start_us":16416,"threads_started":1}
I20260811 00:17:27.340807 5445 tablet_service.cc:1467] Processing CreateTablet for tablet f740d7aef345456a84dc2bb92cfbc8f0 (DEFAULT_TABLE table=test-table [id=5b9e58dafb7c4efcb45d82768ca96de2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:17:27.342468 5445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f740d7aef345456a84dc2bb92cfbc8f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:27.344520 6084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.327955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:41083 (local address 127.0.0.1:39854)
0811 00:17:27.335014 (+ 7059us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:27.335031 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:27.335127 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:27.337593 (+ 2466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:27.337599 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:27.337625 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:27.337840 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:27.337857 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.340030 (+ 2173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.340036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:27.341902 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:27.341919 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.343739 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.343745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:27.343916 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.343949 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 00:17:27.344020 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 00:17:27.344171 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6978,"mutex_wait_us":1,"thread_start_us":8417,"threads_started":1}
I20260811 00:17:27.345856 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.322024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39854 (local address 127.7.94.3:41083)
0811 00:17:27.336997 (+ 14973us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:27.337005 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:27.337036 (+ 31us) server_negotiation.cc:407] Connection header received
0811 00:17:27.337122 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:27.337128 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:27.337391 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:27.338028 (+ 637us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:27.338315 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.339829 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.342161 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.343558 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.344775 (+ 1217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.345004 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:27.345317 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 00:17:27.345503 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14786,"thread_start_us":19302,"threads_started":1}
I20260811 00:17:27.346916 6083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.303257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:37519 (local address 127.0.0.1:57622)
0811 00:17:27.315389 (+ 12132us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:27.315442 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:27.315620 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:27.333072 (+ 17452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:27.333080 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:27.333128 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:27.333724 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:27.333740 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.337865 (+ 4125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.337911 (+ 46us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:27.341973 (+ 4062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:27.341992 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.345674 (+ 3682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.345683 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:27.346025 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.346079 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 00:17:27.346153 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 00:17:27.346397 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11863,"mutex_wait_us":1296,"thread_start_us":12579,"threads_started":1}
I20260811 00:17:27.348661 6085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.303490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57622 (local address 127.7.94.2:37519)
0811 00:17:27.332363 (+ 28873us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:27.332375 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:27.332409 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:17:27.332521 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:27.332532 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:27.332823 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:27.333047 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:27.333984 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.337662 (+ 3678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.343383 (+ 5721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.345404 (+ 2021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.347739 (+ 2335us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.347983 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:27.348057 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 00:17:27.348256 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28667,"thread_start_us":28524,"threads_started":1}
I20260811 00:17:27.349675 5936 tablet_service.cc:1467] Processing CreateTablet for tablet f740d7aef345456a84dc2bb92cfbc8f0 (DEFAULT_TABLE table=test-table [id=5b9e58dafb7c4efcb45d82768ca96de2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:17:27.351116 5936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f740d7aef345456a84dc2bb92cfbc8f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:27.352664 5684 tablet_service.cc:1467] Processing CreateTablet for tablet f740d7aef345456a84dc2bb92cfbc8f0 (DEFAULT_TABLE table=test-table [id=5b9e58dafb7c4efcb45d82768ca96de2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:17:27.354187 5684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f740d7aef345456a84dc2bb92cfbc8f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:27.364753 6093 tablet_bootstrap.cc:492] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd: Bootstrap starting.
I20260811 00:17:27.371628 6093 tablet_bootstrap.cc:654] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:27.373415 6093 log.cc:824] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:27.374866 6094 tablet_bootstrap.cc:492] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d: Bootstrap starting.
I20260811 00:17:27.378571 6093 tablet_bootstrap.cc:492] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd: No bootstrap required, opened a new log
I20260811 00:17:27.378723 6094 tablet_bootstrap.cc:654] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:27.378854 6093 ts_tablet_manager.cc:1397] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.004s
I20260811 00:17:27.380165 6094 log.cc:824] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:27.386072 6097 tablet_bootstrap.cc:492] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f: Bootstrap starting.
I20260811 00:17:27.389850 6094 tablet_bootstrap.cc:492] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d: No bootstrap required, opened a new log
I20260811 00:17:27.389923 6097 tablet_bootstrap.cc:654] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:27.390182 6094 ts_tablet_manager.cc:1397] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.008s
I20260811 00:17:27.391166 6097 log.cc:824] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:27.394382 6093 raft_consensus.cc:348] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a584b2f08d249d884a832162c4dd57f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37519 } } peers { permanent_uuid: "689a07d20b2f41a19104dedf0a51a2fd" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42233 } } peers { permanent_uuid: "351b38562b5d439184acc6e6abb2219d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 41083 } }
I20260811 00:17:27.394946 6093 raft_consensus.cc:374] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:27.395081 6093 raft_consensus.cc:729] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 689a07d20b2f41a19104dedf0a51a2fd, State: Initialized, Role: FOLLOWER
I20260811 00:17:27.395824 6093 consensus_queue.cc:260] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a584b2f08d249d884a832162c4dd57f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37519 } } peers { permanent_uuid: "689a07d20b2f41a19104dedf0a51a2fd" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42233 } } peers { permanent_uuid: "351b38562b5d439184acc6e6abb2219d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 41083 } }
I20260811 00:17:27.404484 6093 ts_tablet_manager.cc:1428] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd: Time spent starting tablet: real 0.026s user 0.014s sys 0.008s
I20260811 00:17:27.405709 6094 raft_consensus.cc:348] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a584b2f08d249d884a832162c4dd57f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37519 } } peers { permanent_uuid: "689a07d20b2f41a19104dedf0a51a2fd" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42233 } } peers { permanent_uuid: "351b38562b5d439184acc6e6abb2219d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 41083 } }
I20260811 00:17:27.406211 6094 raft_consensus.cc:374] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:27.406319 6094 raft_consensus.cc:729] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 351b38562b5d439184acc6e6abb2219d, State: Initialized, Role: FOLLOWER
I20260811 00:17:27.407039 6094 consensus_queue.cc:260] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a584b2f08d249d884a832162c4dd57f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37519 } } peers { permanent_uuid: "689a07d20b2f41a19104dedf0a51a2fd" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42233 } } peers { permanent_uuid: "351b38562b5d439184acc6e6abb2219d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 41083 } }
I20260811 00:17:27.407545 6097 tablet_bootstrap.cc:492] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f: No bootstrap required, opened a new log
I20260811 00:17:27.408231 6059 heartbeater.cc:503] Master 127.7.94.62:44221 was elected leader, sending a full tablet report...
I20260811 00:17:27.408921 6094 ts_tablet_manager.cc:1428] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d: Time spent starting tablet: real 0.019s user 0.015s sys 0.001s
I20260811 00:17:27.409197 6097 ts_tablet_manager.cc:1397] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.014s
I20260811 00:17:27.416649 6097 raft_consensus.cc:348] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a584b2f08d249d884a832162c4dd57f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37519 } } peers { permanent_uuid: "689a07d20b2f41a19104dedf0a51a2fd" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42233 } } peers { permanent_uuid: "351b38562b5d439184acc6e6abb2219d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 41083 } }
I20260811 00:17:27.417011 6097 raft_consensus.cc:374] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:27.417105 6097 raft_consensus.cc:729] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a584b2f08d249d884a832162c4dd57f, State: Initialized, Role: FOLLOWER
I20260811 00:17:27.417799 6097 consensus_queue.cc:260] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a584b2f08d249d884a832162c4dd57f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37519 } } peers { permanent_uuid: "689a07d20b2f41a19104dedf0a51a2fd" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42233 } } peers { permanent_uuid: "351b38562b5d439184acc6e6abb2219d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 41083 } }
I20260811 00:17:27.418931 5807 heartbeater.cc:503] Master 127.7.94.62:44221 was elected leader, sending a full tablet report...
I20260811 00:17:27.419446 6097 ts_tablet_manager.cc:1428] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f: Time spent starting tablet: real 0.010s user 0.008s sys 0.001s
I20260811 00:17:27.454368 6103 raft_consensus.cc:482] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:17:27.454710 6103 raft_consensus.cc:504] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a584b2f08d249d884a832162c4dd57f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37519 } } peers { permanent_uuid: "689a07d20b2f41a19104dedf0a51a2fd" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42233 } } peers { permanent_uuid: "351b38562b5d439184acc6e6abb2219d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 41083 } }
I20260811 00:17:27.456496 6103 leader_election.cc:288] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a584b2f08d249d884a832162c4dd57f (127.7.94.2:37519), 351b38562b5d439184acc6e6abb2219d (127.7.94.3:41083)
I20260811 00:17:27.458333 6105 raft_consensus.cc:482] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:17:27.458637 6105 raft_consensus.cc:504] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a584b2f08d249d884a832162c4dd57f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37519 } } peers { permanent_uuid: "689a07d20b2f41a19104dedf0a51a2fd" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42233 } } peers { permanent_uuid: "351b38562b5d439184acc6e6abb2219d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 41083 } }
I20260811 00:17:27.460520 6105 leader_election.cc:288] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a584b2f08d249d884a832162c4dd57f (127.7.94.2:37519), 689a07d20b2f41a19104dedf0a51a2fd (127.7.94.1:42233)
W20260811 00:17:27.465811 6060 tablet.cc:2369] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:17:27.467191 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.456896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:57979 (local address 127.7.94.3:41083)
0811 00:17:27.457359 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:27.457364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:27.457943 (+ 579us) server_negotiation.cc:407] Connection header received
0811 00:17:27.458257 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:27.458263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:27.458343 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:27.458431 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:27.459288 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.460409 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.463764 (+ 3355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.465478 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.465608 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.465745 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:27.466566 (+ 821us) server_negotiation.cc:299] Negotiation successful
0811 00:17:27.466766 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269}
I20260811 00:17:27.468045 6085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.457097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:44311 (local address 127.7.94.2:37519)
0811 00:17:27.457384 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:27.457394 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:27.458221 (+ 827us) server_negotiation.cc:407] Connection header received
0811 00:17:27.458490 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:27.458496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:27.458577 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:27.458679 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:27.459353 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.460168 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.463416 (+ 3248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.465133 (+ 1717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.465299 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.465432 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:27.467527 (+ 2095us) server_negotiation.cc:299] Negotiation successful
0811 00:17:27.467688 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 00:17:27.468307 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.456768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:41083 (local address 127.7.94.1:57979)
0811 00:17:27.457829 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:27.457844 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:27.458070 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:27.458566 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:27.458571 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:27.458604 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:27.459115 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:27.459132 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.460766 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.460772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:27.463524 (+ 2752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:27.463546 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.466017 (+ 2471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.466028 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:27.466300 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.466371 (+ 71us) client_negotiation.cc:769] Sending connection context
0811 00:17:27.467450 (+ 1079us) client_negotiation.cc:240] Negotiation successful
0811 00:17:27.467645 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":957,"mutex_wait_us":5,"thread_start_us":729,"threads_started":1}
I20260811 00:17:27.469075 6118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.456967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:37519 (local address 127.7.94.1:44311)
0811 00:17:27.458107 (+ 1140us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:27.458121 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:27.458325 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:27.458710 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:27.458715 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:27.458754 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:27.459183 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:27.459196 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.460300 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.460306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:27.463175 (+ 2869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:27.463197 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.467021 (+ 3824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.467030 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:27.467280 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.467342 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 00:17:27.468551 (+ 1209us) client_negotiation.cc:240] Negotiation successful
0811 00:17:27.468695 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1028,"mutex_wait_us":1,"thread_start_us":853,"threads_started":1}
I20260811 00:17:27.469276 5966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f740d7aef345456a84dc2bb92cfbc8f0" candidate_uuid: "689a07d20b2f41a19104dedf0a51a2fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "351b38562b5d439184acc6e6abb2219d" is_pre_election: true
I20260811 00:17:27.469823 5966 raft_consensus.cc:2454] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 689a07d20b2f41a19104dedf0a51a2fd in term 0.
I20260811 00:17:27.470511 5364 leader_election.cc:302] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 351b38562b5d439184acc6e6abb2219d, 689a07d20b2f41a19104dedf0a51a2fd; no voters:
I20260811 00:17:27.470932 6103 raft_consensus.cc:2781] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:17:27.471081 6103 raft_consensus.cc:482] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:17:27.471185 6103 raft_consensus.cc:3037] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:27.474539 6103 raft_consensus.cc:504] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a584b2f08d249d884a832162c4dd57f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37519 } } peers { permanent_uuid: "689a07d20b2f41a19104dedf0a51a2fd" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42233 } } peers { permanent_uuid: "351b38562b5d439184acc6e6abb2219d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 41083 } }
I20260811 00:17:27.475451 6103 leader_election.cc:288] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [CANDIDATE]: Term 1 election: Requested vote from peers 4a584b2f08d249d884a832162c4dd57f (127.7.94.2:37519), 351b38562b5d439184acc6e6abb2219d (127.7.94.3:41083)
I20260811 00:17:27.476089 5966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f740d7aef345456a84dc2bb92cfbc8f0" candidate_uuid: "689a07d20b2f41a19104dedf0a51a2fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "351b38562b5d439184acc6e6abb2219d"
I20260811 00:17:27.476059 5719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f740d7aef345456a84dc2bb92cfbc8f0" candidate_uuid: "689a07d20b2f41a19104dedf0a51a2fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a584b2f08d249d884a832162c4dd57f" is_pre_election: true
I20260811 00:17:27.476392 6063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.460702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:37519 (local address 127.7.94.3:46123)
0811 00:17:27.461128 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:27.461144 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:27.461660 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:27.468680 (+ 7020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:27.468687 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:27.468719 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:27.468919 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:27.468934 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.470546 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.470551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:27.472613 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:27.472630 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.475474 (+ 2844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.475481 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:27.475712 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.475749 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 00:17:27.475855 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 00:17:27.475984 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":3}
I20260811 00:17:27.476325 5966 raft_consensus.cc:3037] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:27.476682 5719 raft_consensus.cc:2454] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 689a07d20b2f41a19104dedf0a51a2fd in term 0.
I20260811 00:17:27.477293 6085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.461343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:46123 (local address 127.7.94.2:37519)
0811 00:17:27.468309 (+ 6966us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:27.468317 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:27.468338 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:17:27.468405 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:27.468412 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:27.468502 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:27.469057 (+ 555us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:27.469197 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.470396 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.474015 (+ 3619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.475291 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.476713 (+ 1422us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.476821 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:27.476872 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 00:17:27.476997 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6846,"thread_start_us":13808,"threads_started":1}
I20260811 00:17:27.477587 5719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f740d7aef345456a84dc2bb92cfbc8f0" candidate_uuid: "689a07d20b2f41a19104dedf0a51a2fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a584b2f08d249d884a832162c4dd57f"
I20260811 00:17:27.477761 5719 raft_consensus.cc:3037] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:27.478477 5718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f740d7aef345456a84dc2bb92cfbc8f0" candidate_uuid: "351b38562b5d439184acc6e6abb2219d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a584b2f08d249d884a832162c4dd57f" is_pre_election: true
I20260811 00:17:27.479148 6088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.461108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:46571 (local address 127.7.94.1:42233)
0811 00:17:27.461313 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:27.461318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:27.470020 (+ 8702us) server_negotiation.cc:407] Connection header received
0811 00:17:27.470281 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:27.470289 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:27.470379 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:27.470447 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:27.472025 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.473569 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.476593 (+ 3024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.477799 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.477911 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.478002 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:27.478761 (+ 759us) server_negotiation.cc:299] Negotiation successful
0811 00:17:27.478928 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 00:17:27.479979 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.460763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:42233 (local address 127.7.94.3:46571)
0811 00:17:27.469916 (+ 9153us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:27.469938 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:27.470107 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:27.470726 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:27.470731 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:27.470759 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:27.471783 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:27.471803 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.473837 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.473845 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:27.476284 (+ 2439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:27.476301 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.478337 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.478347 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:27.478533 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.478574 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 00:17:27.479420 (+ 846us) client_negotiation.cc:240] Negotiation successful
0811 00:17:27.479572 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8978,"thread_start_us":8717,"threads_started":1}
I20260811 00:17:27.480129 5966 raft_consensus.cc:2454] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 689a07d20b2f41a19104dedf0a51a2fd in term 1.
I20260811 00:17:27.480660 5364 leader_election.cc:302] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 351b38562b5d439184acc6e6abb2219d, 689a07d20b2f41a19104dedf0a51a2fd; no voters:
I20260811 00:17:27.480952 6103 raft_consensus.cc:2781] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:17:27.481110 5719 raft_consensus.cc:2454] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 689a07d20b2f41a19104dedf0a51a2fd in term 1.
I20260811 00:17:27.480990 5483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f740d7aef345456a84dc2bb92cfbc8f0" candidate_uuid: "351b38562b5d439184acc6e6abb2219d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "689a07d20b2f41a19104dedf0a51a2fd" is_pre_election: true
I20260811 00:17:27.481576 6103 raft_consensus.cc:686] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [term 1 LEADER]: Becoming Leader. State: Replica: 689a07d20b2f41a19104dedf0a51a2fd, State: Running, Role: LEADER
I20260811 00:17:27.482067 6103 consensus_queue.cc:237] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a584b2f08d249d884a832162c4dd57f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37519 } } peers { permanent_uuid: "689a07d20b2f41a19104dedf0a51a2fd" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42233 } } peers { permanent_uuid: "351b38562b5d439184acc6e6abb2219d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 41083 } }
I20260811 00:17:27.485647 5845 leader_election.cc:302] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 351b38562b5d439184acc6e6abb2219d; no voters: 4a584b2f08d249d884a832162c4dd57f, 689a07d20b2f41a19104dedf0a51a2fd
I20260811 00:17:27.486052 6105 raft_consensus.cc:2726] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 00:17:27.486886 5219 catalog_manager.cc:5495] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 689a07d20b2f41a19104dedf0a51a2fd (127.7.94.1). New cstate: current_term: 1 leader_uuid: "689a07d20b2f41a19104dedf0a51a2fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a584b2f08d249d884a832162c4dd57f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "689a07d20b2f41a19104dedf0a51a2fd" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 42233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "351b38562b5d439184acc6e6abb2219d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 41083 } health_report { overall_health: UNKNOWN } } }
I20260811 00:17:27.522039 6088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.517148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38114 (local address 127.7.94.1:42233)
0811 00:17:27.517338 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:27.517343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:27.517358 (+ 15us) server_negotiation.cc:407] Connection header received
0811 00:17:27.517439 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:27.517443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:27.517532 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:27.517679 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:17:27.518009 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.518711 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.520061 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.520420 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.520512 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.520937 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:17:27.521517 (+ 580us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:17:27.521629 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:27.521748 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 00:17:27.521813 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 00:17:27.530229 5719 raft_consensus.cc:1261] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f [term 1 FOLLOWER]: Refusing update from remote peer 689a07d20b2f41a19104dedf0a51a2fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:17:27.530295 5966 raft_consensus.cc:1261] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d [term 1 FOLLOWER]: Refusing update from remote peer 689a07d20b2f41a19104dedf0a51a2fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:17:27.531229 6122 consensus_queue.cc:1035] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "351b38562b5d439184acc6e6abb2219d" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 41083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:17:27.531560 6103 consensus_queue.cc:1035] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a584b2f08d249d884a832162c4dd57f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 37519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:17:27.544806 6126 mvcc.cc:205] Tried to move back new op lower bound from 7317124905074507776 to 7317124904890134528. Current Snapshot: MvccSnapshot[applied={T|T < 7317124905074507776}]
I20260811 00:17:27.558415 5220 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:17:27.558684 5220 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:17:27.562201 5220 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb10f164914d40d5ade7932634cddec7: Sending DeleteTablet for 3 replicas of tablet f740d7aef345456a84dc2bb92cfbc8f0
I20260811 00:17:27.563360 5684 tablet_service.cc:1514] Processing DeleteTablet for tablet f740d7aef345456a84dc2bb92cfbc8f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:17:27 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:57622
I20260811 00:17:27.563474 5445 tablet_service.cc:1514] Processing DeleteTablet for tablet f740d7aef345456a84dc2bb92cfbc8f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:17:27 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:38110
I20260811 00:17:27.563675 5936 tablet_service.cc:1514] Processing DeleteTablet for tablet f740d7aef345456a84dc2bb92cfbc8f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:17:27 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:39854
I20260811 00:17:27.565876 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 5334
I20260811 00:17:27.574244 6146 tablet_replica.cc:330] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d: stopping tablet replica
I20260811 00:17:27.574738 6146 raft_consensus.cc:2229] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:17:27.575269 6146 raft_consensus.cc:2258] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:17:27.577782 6146 ts_tablet_manager.cc:1905] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:17:27.581921 6145 tablet_replica.cc:330] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd: stopping tablet replica
I20260811 00:17:27.582435 6145 raft_consensus.cc:2229] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:17:27.583244 6145 raft_consensus.cc:2258] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:17:27.587134 6145 ts_tablet_manager.cc:1905] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:17:27.589005 6146 ts_tablet_manager.cc:1918] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:17:27.589260 6146 log.cc:1196] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/f740d7aef345456a84dc2bb92cfbc8f0
I20260811 00:17:27.589874 6146 ts_tablet_manager.cc:1939] T f740d7aef345456a84dc2bb92cfbc8f0 P 351b38562b5d439184acc6e6abb2219d: Deleting consensus metadata
I20260811 00:17:27.591858 5201 catalog_manager.cc:4841] TS 351b38562b5d439184acc6e6abb2219d (127.7.94.3:41083): tablet f740d7aef345456a84dc2bb92cfbc8f0 (table test-table [id=5b9e58dafb7c4efcb45d82768ca96de2]) successfully deleted
I20260811 00:17:27.593791 6143 tablet_replica.cc:330] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f: stopping tablet replica
I20260811 00:17:27.594280 6143 raft_consensus.cc:2229] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:17:27.594802 6143 raft_consensus.cc:2258] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:17:27.595417 6145 ts_tablet_manager.cc:1918] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:17:27.595639 6145 log.cc:1196] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/f740d7aef345456a84dc2bb92cfbc8f0
I20260811 00:17:27.596221 6145 ts_tablet_manager.cc:1939] T f740d7aef345456a84dc2bb92cfbc8f0 P 689a07d20b2f41a19104dedf0a51a2fd: Deleting consensus metadata
I20260811 00:17:27.596804 6143 ts_tablet_manager.cc:1905] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:17:27.598067 5201 catalog_manager.cc:4841] TS 689a07d20b2f41a19104dedf0a51a2fd (127.7.94.1:42233): tablet f740d7aef345456a84dc2bb92cfbc8f0 (table test-table [id=5b9e58dafb7c4efcb45d82768ca96de2]) successfully deleted
Aug 11 00:17:27 dist-test-slave-8hhm krb5kdc[5144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:17:27.602818 6143 ts_tablet_manager.cc:1918] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:17:27.602998 6143 log.cc:1196] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/f740d7aef345456a84dc2bb92cfbc8f0
I20260811 00:17:27.603554 6143 ts_tablet_manager.cc:1939] T f740d7aef345456a84dc2bb92cfbc8f0 P 4a584b2f08d249d884a832162c4dd57f: Deleting consensus metadata
I20260811 00:17:27.605000 5202 catalog_manager.cc:4841] TS 4a584b2f08d249d884a832162c4dd57f (127.7.94.2:37519): tablet f740d7aef345456a84dc2bb92cfbc8f0 (table test-table [id=5b9e58dafb7c4efcb45d82768ca96de2]) successfully deleted
I20260811 00:17:27.610894 6088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.567132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38116 (local address 127.7.94.1:42233)
0811 00:17:27.567383 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:27.567392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:27.591095 (+ 23703us) server_negotiation.cc:407] Connection header received
0811 00:17:27.591606 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:27.591612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:27.591729 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:27.591814 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:27.593430 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.594868 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.597321 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.597978 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.598119 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.603826 (+ 5707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:27.603840 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:27.603857 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:27.603896 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:27.608043 (+ 4147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:27.608695 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:27.608704 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:27.608708 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:27.608875 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:27.609242 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:27.609247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:27.609251 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:27.609751 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:27.610079 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:27.610268 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 00:17:27.610535 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 00:17:27.611748 5557 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:27.711804 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 5334
I20260811 00:17:27.738853 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 5574
Aug 11 00:17:27 dist-test-slave-8hhm krb5kdc[5144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.2@KRBTEST.COM
I20260811 00:17:27.754408 6085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.739745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57626 (local address 127.7.94.2:37519)
0811 00:17:27.739914 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:27.739918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:27.739934 (+ 16us) server_negotiation.cc:407] Connection header received
0811 00:17:27.739989 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:27.739993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:27.740070 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:27.740153 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:27.741453 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.742185 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.743456 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.743843 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.743933 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.748532 (+ 4599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:27.748548 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:27.748564 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:27.748599 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:27.751476 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:27.752217 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:27.752225 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:27.752228 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:27.752361 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:27.752763 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:27.752768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:27.752771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:27.753142 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:27.753507 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:27.753853 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 00:17:27.754087 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 00:17:27.755125 5800 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:27.846437 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 5574
I20260811 00:17:27.874222 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 5817
Aug 11 00:17:27 dist-test-slave-8hhm krb5kdc[5144](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.3@KRBTEST.COM
I20260811 00:17:27.888988 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:27.874973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39860 (local address 127.7.94.3:41083)
0811 00:17:27.875142 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:27.875147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:27.875164 (+ 17us) server_negotiation.cc:407] Connection header received
0811 00:17:27.875324 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:27.875328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:27.875404 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:27.875490 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:27.876768 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.877567 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.878820 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:27.879175 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:27.879258 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:27.883719 (+ 4461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:27.883734 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:27.883750 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:27.883785 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:27.886648 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:27.887276 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:27.887283 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:27.887285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:27.887397 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:27.887705 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:27.887708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:27.887710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:27.888047 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:27.888279 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:27.888559 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 00:17:27.888725 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 00:17:27.889737 6048 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:27.978636 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 5817
I20260811 00:17:28.004405 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 5171
I20260811 00:17:28.005614 5272 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:28.113950 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 5171
2026-08-11T00:17:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4846 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T00:17:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:17:28Z Disabled control of system clock
I20260811 00:17:28.216642 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:41787
--webserver_interface=127.7.94.62
--webserver_port=0
--builtin_ntp_servers=127.7.94.20:35709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:41787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 00:17:28.552757 6161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:28.553067 6161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:28.553157 6161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:28.560230 6161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:17:28.560325 6161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:28.560364 6161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:28.560420 6161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:17:28.560462 6161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:17:28.569283 6161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:35709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:41787
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:41787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:28.570482 6161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:28.572278 6161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:28.583844 6174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:28.583906 6171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:28.584133 6178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:28.584808 6161 server_base.cc:1034] running on GCE node
I20260811 00:17:28.585803 6161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:28.587262 6161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:28.588539 6161 hybrid_clock.cc:648] HybridClock initialized: now 1786407448588486 us; error 33 us; skew 500 ppm
I20260811 00:17:28.588938 6161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:17:28.595160 6161 webserver.cc:460] Webserver started at http://127.7.94.62:46149/ using document root <none> and password file <none>
I20260811 00:17:28.595919 6161 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:28.596037 6161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:28.596385 6161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:28.599303 6161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "be65352ccd4745a6ac817af9c26118a5"
format_stamp: "Formatted at 2026-08-11 00:17:28 on dist-test-slave-8hhm"
server_key: "6f034f78f83bfa1a1e2ca14f4ff72172"
server_key_iv: "205f84a141107b9eaf4cdca545d0e030"
server_key_version: "encryptionkey@0"
I20260811 00:17:28.600142 6161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "be65352ccd4745a6ac817af9c26118a5"
format_stamp: "Formatted at 2026-08-11 00:17:28 on dist-test-slave-8hhm"
server_key: "6f034f78f83bfa1a1e2ca14f4ff72172"
server_key_iv: "205f84a141107b9eaf4cdca545d0e030"
server_key_version: "encryptionkey@0"
I20260811 00:17:28.605716 6161 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 00:17:28.608961 6188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:28.610307 6161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:17:28.610526 6161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "be65352ccd4745a6ac817af9c26118a5"
format_stamp: "Formatted at 2026-08-11 00:17:28 on dist-test-slave-8hhm"
server_key: "6f034f78f83bfa1a1e2ca14f4ff72172"
server_key_iv: "205f84a141107b9eaf4cdca545d0e030"
server_key_version: "encryptionkey@0"
I20260811 00:17:28.610755 6161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:28.631482 6161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:28.632325 6161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:28.632624 6161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:28.650950 6161 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:41787
I20260811 00:17:28.650969 6286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:41787 every 8 connection(s)
I20260811 00:17:28.652274 6161 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:17:28.653594 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 6161
I20260811 00:17:28.653769 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:17:28.654328 7544 external_mini_cluster.cc:1428] Setting key 45296552d211d03034068b6565dd0b58
I20260811 00:17:28.656745 6288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:28.667873 6295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:28.656130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57420 (local address 127.7.94.62:41787)
0811 00:17:28.657191 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:28.657203 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:28.657299 (+ 96us) server_negotiation.cc:407] Connection header received
0811 00:17:28.658841 (+ 1542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:28.658861 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:28.659326 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:28.659662 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:28.660241 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:28.661573 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:28.663584 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:28.664224 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:28.664354 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:28.665632 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:28.665640 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:28.665656 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:28.666185 (+ 529us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:28.666255 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:28.666289 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:28.666378 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:28.666621 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 00:17:28.666946 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":370,"threads_started":1}
I20260811 00:17:28.669178 6288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5: Bootstrap starting.
I20260811 00:17:28.672333 6288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:28.673439 6288 log.cc:824] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:28.675927 6288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5: No bootstrap required, opened a new log
I20260811 00:17:28.682964 6288 raft_consensus.cc:348] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be65352ccd4745a6ac817af9c26118a5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41787 } }
I20260811 00:17:28.683386 6288 raft_consensus.cc:374] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:28.683486 6288 raft_consensus.cc:729] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be65352ccd4745a6ac817af9c26118a5, State: Initialized, Role: FOLLOWER
I20260811 00:17:28.683981 6288 consensus_queue.cc:260] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be65352ccd4745a6ac817af9c26118a5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41787 } }
I20260811 00:17:28.684211 6288 raft_consensus.cc:388] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:17:28.684379 6288 raft_consensus.cc:482] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:17:28.684566 6288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:28.686738 6288 raft_consensus.cc:504] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be65352ccd4745a6ac817af9c26118a5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41787 } }
I20260811 00:17:28.687202 6288 leader_election.cc:302] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be65352ccd4745a6ac817af9c26118a5; no voters:
I20260811 00:17:28.687881 6288 leader_election.cc:288] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:17:28.687995 6297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:17:28.688704 6297 raft_consensus.cc:686] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5 [term 1 LEADER]: Becoming Leader. State: Replica: be65352ccd4745a6ac817af9c26118a5, State: Running, Role: LEADER
I20260811 00:17:28.689251 6297 consensus_queue.cc:237] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be65352ccd4745a6ac817af9c26118a5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41787 } }
I20260811 00:17:28.690052 6288 sys_catalog.cc:564] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:17:28.692059 6299 sys_catalog.cc:455] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be65352ccd4745a6ac817af9c26118a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be65352ccd4745a6ac817af9c26118a5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41787 } } }
I20260811 00:17:28.692484 6299 sys_catalog.cc:458] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5 [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:28.692963 6300 sys_catalog.cc:455] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be65352ccd4745a6ac817af9c26118a5. Latest consensus state: current_term: 1 leader_uuid: "be65352ccd4745a6ac817af9c26118a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be65352ccd4745a6ac817af9c26118a5" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41787 } } }
I20260811 00:17:28.693585 6300 sys_catalog.cc:458] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5 [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:28.696388 6310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:17:28.700526 6310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:17:28.708475 6310 catalog_manager.cc:1347] Generated new cluster ID: 941d259de7ac4666838a928a771ecad8
I20260811 00:17:28.708591 6310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:17:28.719590 6310 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:17:28.720813 6310 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:17:28.733472 6310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5: Generated new TSK 0
I20260811 00:17:28.734475 6310 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:17:28.745767 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41787
--builtin_ntp_servers=127.7.94.20:35709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 00:17:29.089190 6324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:29.089536 6324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:29.089697 6324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:29.096998 6324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:29.097131 6324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:29.097337 6324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:17:29.106212 6324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:35709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:29.107437 6324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:29.109227 6324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:29.122478 6338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:29.122539 6344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:29.122551 6334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:29.123188 6324 server_base.cc:1034] running on GCE node
I20260811 00:17:29.123816 6324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:29.124696 6324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:29.125921 6324 hybrid_clock.cc:648] HybridClock initialized: now 1786407449125877 us; error 34 us; skew 500 ppm
I20260811 00:17:29.126219 6324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:17:29.128479 6324 webserver.cc:460] Webserver started at http://127.7.94.1:33469/ using document root <none> and password file <none>
I20260811 00:17:29.129141 6324 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:29.129293 6324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:29.129628 6324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:29.132436 6324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "aaf78c04d38e4dc99ceec0e15d6395c4"
format_stamp: "Formatted at 2026-08-11 00:17:29 on dist-test-slave-8hhm"
server_key: "44ead6b51e417a71c33804be27827ad8"
server_key_iv: "18b12f0ee43974a2b2d40eabd85e67b4"
server_key_version: "encryptionkey@0"
I20260811 00:17:29.133332 6324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "aaf78c04d38e4dc99ceec0e15d6395c4"
format_stamp: "Formatted at 2026-08-11 00:17:29 on dist-test-slave-8hhm"
server_key: "44ead6b51e417a71c33804be27827ad8"
server_key_iv: "18b12f0ee43974a2b2d40eabd85e67b4"
server_key_version: "encryptionkey@0"
I20260811 00:17:29.138442 6324 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260811 00:17:29.141960 6355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:29.143160 6324 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 00:17:29.143338 6324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "aaf78c04d38e4dc99ceec0e15d6395c4"
format_stamp: "Formatted at 2026-08-11 00:17:29 on dist-test-slave-8hhm"
server_key: "44ead6b51e417a71c33804be27827ad8"
server_key_iv: "18b12f0ee43974a2b2d40eabd85e67b4"
server_key_version: "encryptionkey@0"
I20260811 00:17:29.143585 6324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:29.166313 6324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:29.167127 6324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:29.167410 6324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:29.168427 6324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:29.170014 6324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:17:29.170137 6324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:29.170264 6324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:29.170353 6324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:29.204594 6324 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:36937
I20260811 00:17:29.204643 6567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:36937 every 8 connection(s)
I20260811 00:17:29.206007 6324 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:17:29.207458 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 6324
I20260811 00:17:29.207646 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:17:29.208132 7544 external_mini_cluster.cc:1428] Setting key 6ec0fc9f346b505be9122e940da850f2
I20260811 00:17:29.211349 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41787
--builtin_ntp_servers=127.7.94.20:35709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 00:17:29.221055 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:29.208504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:56037 (local address 127.7.94.62:41787)
0811 00:17:29.208947 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:29.208954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:29.210833 (+ 1879us) server_negotiation.cc:407] Connection header received
0811 00:17:29.213668 (+ 2835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:29.213673 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:29.213733 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:29.213823 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:29.215201 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:29.216044 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:29.218350 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:29.218718 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:29.218805 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:29.220363 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:29.220367 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:29.220370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:29.220386 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:29.220424 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:29.220436 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:29.220501 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:29.220753 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 00:17:29.220824 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":219,"threads_started":1}
I20260811 00:17:29.222509 6575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:29.209176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41787 (local address 127.7.94.1:56037)
0811 00:17:29.210673 (+ 1497us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:29.210722 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:29.213387 (+ 2665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:29.214076 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:29.214088 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:29.214211 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:17:29.214996 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:29.215022 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:29.216208 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:29.216216 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:29.218219 (+ 2003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:29.218237 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:29.218941 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:29.218950 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:29.219253 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:29.219996 (+ 743us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:17:29.220002 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:17:29.220066 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:17:29.220168 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:17:29.220536 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:17:29.220550 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:17:29.220560 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 00:17:29.220736 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 00:17:29.221126 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":905,"thread_start_us":644,"threads_started":1}
I20260811 00:17:29.224061 6568 heartbeater.cc:347] Connected to a master server at 127.7.94.62:41787
I20260811 00:17:29.224357 6568 heartbeater.cc:464] Registering TS with master...
I20260811 00:17:29.225028 6568 heartbeater.cc:511] Master 127.7.94.62:41787 requested a full tablet report, sending...
I20260811 00:17:29.227214 6224 ts_manager.cc:194] Registered new tserver with Master: aaf78c04d38e4dc99ceec0e15d6395c4 (127.7.94.1:36937)
I20260811 00:17:29.229458 6224 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.94.1:56037
W20260811 00:17:29.551142 6577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:29.551434 6577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:29.551522 6577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:29.558719 6577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:29.558816 6577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:29.558915 6577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:17:29.567909 6577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:35709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:29.569130 6577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:29.570925 6577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:29.584302 6594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:29.584312 6591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:29.584249 6592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:29.584909 6577 server_base.cc:1034] running on GCE node
I20260811 00:17:29.585695 6577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:29.586570 6577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:29.587772 6577 hybrid_clock.cc:648] HybridClock initialized: now 1786407449587717 us; error 49 us; skew 500 ppm
I20260811 00:17:29.588101 6577 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:17:29.590463 6577 webserver.cc:460] Webserver started at http://127.7.94.2:46777/ using document root <none> and password file <none>
I20260811 00:17:29.591193 6577 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:29.591328 6577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:29.591686 6577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:29.594769 6577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "d1b62abe883a4e2f960e415cc90f5db3"
format_stamp: "Formatted at 2026-08-11 00:17:29 on dist-test-slave-8hhm"
server_key: "09cebd6122081fd5653a966e8619cb34"
server_key_iv: "eb0036e4c6a417aaa253fa632499cca9"
server_key_version: "encryptionkey@0"
I20260811 00:17:29.595592 6577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "d1b62abe883a4e2f960e415cc90f5db3"
format_stamp: "Formatted at 2026-08-11 00:17:29 on dist-test-slave-8hhm"
server_key: "09cebd6122081fd5653a966e8619cb34"
server_key_iv: "eb0036e4c6a417aaa253fa632499cca9"
server_key_version: "encryptionkey@0"
I20260811 00:17:29.600512 6577 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 00:17:29.603886 6601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:29.605111 6577 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 00:17:29.605305 6577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "d1b62abe883a4e2f960e415cc90f5db3"
format_stamp: "Formatted at 2026-08-11 00:17:29 on dist-test-slave-8hhm"
server_key: "09cebd6122081fd5653a966e8619cb34"
server_key_iv: "eb0036e4c6a417aaa253fa632499cca9"
server_key_version: "encryptionkey@0"
I20260811 00:17:29.605513 6577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:29.622293 6577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:29.623064 6577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:29.623351 6577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:29.624296 6577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:29.625756 6577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:17:29.625862 6577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:29.625995 6577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:29.626101 6577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:29.661046 6577 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:36569
I20260811 00:17:29.661144 6798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:36569 every 8 connection(s)
I20260811 00:17:29.662355 6577 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:17:29.665553 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 6577
I20260811 00:17:29.665740 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:17:29.666385 7544 external_mini_cluster.cc:1428] Setting key 23e4974b082235ff4f10bc44ac33e11e
I20260811 00:17:29.669345 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41787
--builtin_ntp_servers=127.7.94.20:35709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 00:17:29.676003 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:29.664970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:39865 (local address 127.7.94.62:41787)
0811 00:17:29.665176 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:29.665184 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:29.666899 (+ 1715us) server_negotiation.cc:407] Connection header received
0811 00:17:29.668974 (+ 2075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:29.668982 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:29.669066 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:29.669194 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:29.670298 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:29.671482 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:29.673034 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:29.673705 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:29.673863 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:29.675128 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:29.675137 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:29.675144 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:29.675175 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:29.675216 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:29.675234 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:29.675326 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:29.675521 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 00:17:29.675627 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 00:17:29.676744 6803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:29.665588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41787 (local address 127.7.94.2:39865)
0811 00:17:29.666754 (+ 1166us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:29.666798 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:29.668718 (+ 1920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:29.669454 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:29.669467 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:29.669590 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:17:29.670121 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:29.670136 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:29.671655 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:29.671659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:29.672876 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:29.672889 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:29.673932 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:29.673937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:29.674129 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:29.674831 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:17:29.674834 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:17:29.674870 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:17:29.674944 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:17:29.675331 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:17:29.675339 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:17:29.675345 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 00:17:29.675476 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 00:17:29.675745 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":349,"threads_started":1}
I20260811 00:17:29.678120 6799 heartbeater.cc:347] Connected to a master server at 127.7.94.62:41787
I20260811 00:17:29.678449 6799 heartbeater.cc:464] Registering TS with master...
I20260811 00:17:29.679282 6799 heartbeater.cc:511] Master 127.7.94.62:41787 requested a full tablet report, sending...
I20260811 00:17:29.680750 6224 ts_manager.cc:194] Registered new tserver with Master: d1b62abe883a4e2f960e415cc90f5db3 (127.7.94.2:36569)
I20260811 00:17:29.681591 6224 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.94.2:39865
W20260811 00:17:30.003158 6806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:30.003465 6806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:30.003582 6806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:30.011050 6806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:30.011166 6806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:30.011308 6806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:17:30.020606 6806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:35709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:30.021908 6806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:30.023815 6806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:30.037061 6814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:30.037036 6816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:30.036981 6818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:30.037850 6806 server_base.cc:1034] running on GCE node
I20260811 00:17:30.038492 6806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:30.039407 6806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:30.040643 6806 hybrid_clock.cc:648] HybridClock initialized: now 1786407450040594 us; error 40 us; skew 500 ppm
I20260811 00:17:30.041014 6806 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:17:30.043318 6806 webserver.cc:460] Webserver started at http://127.7.94.3:37107/ using document root <none> and password file <none>
I20260811 00:17:30.043984 6806 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:30.044114 6806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:30.044448 6806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:30.047508 6806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "f61580d4df6340fc96575786e1864861"
format_stamp: "Formatted at 2026-08-11 00:17:30 on dist-test-slave-8hhm"
server_key: "9d91efd71d61c9b76ccf8a6edcdd44e4"
server_key_iv: "13c0fe28653879e6e93a2cd436ebd08c"
server_key_version: "encryptionkey@0"
I20260811 00:17:30.048434 6806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "f61580d4df6340fc96575786e1864861"
format_stamp: "Formatted at 2026-08-11 00:17:30 on dist-test-slave-8hhm"
server_key: "9d91efd71d61c9b76ccf8a6edcdd44e4"
server_key_iv: "13c0fe28653879e6e93a2cd436ebd08c"
server_key_version: "encryptionkey@0"
I20260811 00:17:30.053257 6806 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 00:17:30.056577 6828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:30.057886 6806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:17:30.058059 6806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "f61580d4df6340fc96575786e1864861"
format_stamp: "Formatted at 2026-08-11 00:17:30 on dist-test-slave-8hhm"
server_key: "9d91efd71d61c9b76ccf8a6edcdd44e4"
server_key_iv: "13c0fe28653879e6e93a2cd436ebd08c"
server_key_version: "encryptionkey@0"
I20260811 00:17:30.058274 6806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:30.090555 6806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:30.091424 6806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:30.091732 6806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:30.092729 6806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:30.094340 6806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:17:30.094451 6806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:30.094581 6806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:30.094673 6806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:30.127588 6806 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:40785
I20260811 00:17:30.127686 7032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:40785 every 8 connection(s)
I20260811 00:17:30.128921 6806 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:17:30.130898 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 6806
I20260811 00:17:30.131072 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:17:30.131618 7544 external_mini_cluster.cc:1428] Setting key b7bbc5fd374be39d46e5a044f6f76ece
I20260811 00:17:30.140847 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.131737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:53913 (local address 127.7.94.62:41787)
0811 00:17:30.131983 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:30.131990 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:30.133582 (+ 1592us) server_negotiation.cc:407] Connection header received
0811 00:17:30.135284 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:30.135288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:30.135350 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:30.135428 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:30.136382 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.137110 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.138589 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.138989 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.139074 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:30.140194 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:30.140199 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:30.140202 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:30.140218 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:30.140247 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:30.140259 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:30.140325 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:30.140515 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 00:17:30.140585 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 00:17:30.141803 7038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.132339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41787 (local address 127.7.94.3:53913)
0811 00:17:30.133455 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:30.133499 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:30.135057 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:30.135622 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:30.135629 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:30.135716 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:17:30.136250 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:30.136265 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.137289 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.137292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:30.138468 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:30.138479 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.139152 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.139156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:30.139328 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:30.139927 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:17:30.139930 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:17:30.139964 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:17:30.140036 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:17:30.140345 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:17:30.140351 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:17:30.140357 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 00:17:30.140485 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 00:17:30.140746 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":352,"threads_started":1}
I20260811 00:17:30.143086 7035 heartbeater.cc:347] Connected to a master server at 127.7.94.62:41787
I20260811 00:17:30.143358 7035 heartbeater.cc:464] Registering TS with master...
I20260811 00:17:30.144042 7035 heartbeater.cc:511] Master 127.7.94.62:41787 requested a full tablet report, sending...
I20260811 00:17:30.145486 6224 ts_manager.cc:194] Registered new tserver with Master: f61580d4df6340fc96575786e1864861 (127.7.94.3:40785)
I20260811 00:17:30.146417 6224 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.94.3:53913
I20260811 00:17:30.146696 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:17:30.156764 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.151366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57428 (local address 127.7.94.62:41787)
0811 00:17:30.151587 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:30.151594 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:30.151878 (+ 284us) server_negotiation.cc:407] Connection header received
0811 00:17:30.151996 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:30.152000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:30.152068 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:30.152146 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:30.152651 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.153374 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.154676 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.155074 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.155164 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:30.156118 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:30.156122 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:30.156125 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:30.156140 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:30.156169 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:30.156193 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:30.156258 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:30.156374 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 00:17:30.156438 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 00:17:30.160108 6224 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57428:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:17:30.167230 6224 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:17:30.191612 7063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.179741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:36569 (local address 127.0.0.1:56926)
0811 00:17:30.182193 (+ 2452us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:30.182217 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:30.182425 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:30.183717 (+ 1292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:30.183724 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:30.183763 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:30.184173 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:30.184183 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.186304 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.186309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:30.188682 (+ 2373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:30.188697 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.190690 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.190697 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:30.190855 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:30.190887 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 00:17:30.190949 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 00:17:30.191181 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2273,"mutex_wait_us":105,"thread_start_us":1341,"threads_started":1}
I20260811 00:17:30.191813 7065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.181523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36142 (local address 127.7.94.3:40785)
0811 00:17:30.183385 (+ 1862us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:30.183392 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:30.183962 (+ 570us) server_negotiation.cc:407] Connection header received
0811 00:17:30.184142 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:30.184149 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:30.184353 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:30.184541 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:30.185208 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.186771 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.189365 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.190360 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.190494 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:30.190771 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:30.191173 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 00:17:30.191388 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1716,"thread_start_us":1569,"threads_started":1}
I20260811 00:17:30.191637 7058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.181175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:40785 (local address 127.0.0.1:36142)
0811 00:17:30.183833 (+ 2658us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:30.183851 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:30.183978 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:30.184752 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:30.184755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:30.184772 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:30.185015 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:30.185022 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.186944 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.186951 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:30.189154 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:30.189165 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.190547 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.190554 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:30.190743 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:30.190801 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 00:17:30.190942 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 00:17:30.191195 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2538,"mutex_wait_us":74,"thread_start_us":637,"threads_started":1}
I20260811 00:17:30.192771 7062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.180282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56926 (local address 127.7.94.2:36569)
0811 00:17:30.183027 (+ 2745us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:30.183037 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:30.183069 (+ 32us) server_negotiation.cc:407] Connection header received
0811 00:17:30.183194 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:30.183203 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:30.183438 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:30.184359 (+ 921us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:30.184553 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.186113 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.189010 (+ 2897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.190483 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.191992 (+ 1509us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:30.192215 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:30.192287 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 00:17:30.192442 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2576,"thread_start_us":2468,"threads_started":1}
I20260811 00:17:30.192966 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.179018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:36937 (local address 127.0.0.1:34096)
0811 00:17:30.181518 (+ 2500us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:30.181555 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:30.182953 (+ 1398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:30.183850 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:30.183855 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:30.183888 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:30.184382 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:30.184397 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.186292 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.186298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:30.188888 (+ 2590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:30.188903 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.192210 (+ 3307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.192217 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:30.192393 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:30.192424 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 00:17:30.192507 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 00:17:30.192627 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2235,"mutex_wait_us":182,"thread_start_us":4532,"threads_started":1}
I20260811 00:17:30.193804 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.179169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34096 (local address 127.7.94.1:36937)
0811 00:17:30.180199 (+ 1030us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:30.180211 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:30.182820 (+ 2609us) server_negotiation.cc:407] Connection header received
0811 00:17:30.183168 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:30.183177 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:30.183455 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:30.183636 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:30.184564 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.186119 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.189161 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.191632 (+ 2471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.191820 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:30.192063 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:30.193221 (+ 1158us) server_negotiation.cc:299] Negotiation successful
0811 00:17:30.193402 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":804,"thread_start_us":649,"threads_started":1}
I20260811 00:17:30.196405 6923 tablet_service.cc:1467] Processing CreateTablet for tablet 60415bbd72da4d57a4d8edbb45de7397 (DEFAULT_TABLE table=test-table [id=ca89f5b2c185466ba235d63f0cebdf55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:17:30.196568 6685 tablet_service.cc:1467] Processing CreateTablet for tablet 60415bbd72da4d57a4d8edbb45de7397 (DEFAULT_TABLE table=test-table [id=ca89f5b2c185466ba235d63f0cebdf55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:17:30.196671 6444 tablet_service.cc:1467] Processing CreateTablet for tablet 60415bbd72da4d57a4d8edbb45de7397 (DEFAULT_TABLE table=test-table [id=ca89f5b2c185466ba235d63f0cebdf55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:17:30.197862 6444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60415bbd72da4d57a4d8edbb45de7397. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:30.197921 6685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60415bbd72da4d57a4d8edbb45de7397. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:30.197921 6923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60415bbd72da4d57a4d8edbb45de7397. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:30.216652 7066 tablet_bootstrap.cc:492] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4: Bootstrap starting.
I20260811 00:17:30.218057 7070 tablet_bootstrap.cc:492] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861: Bootstrap starting.
I20260811 00:17:30.220649 7066 tablet_bootstrap.cc:654] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:30.221089 7069 tablet_bootstrap.cc:492] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3: Bootstrap starting.
I20260811 00:17:30.222245 7070 tablet_bootstrap.cc:654] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:30.222437 7066 log.cc:824] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:30.223445 7070 log.cc:824] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:30.224777 7069 tablet_bootstrap.cc:654] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:30.226014 7069 log.cc:824] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:30.226138 7070 tablet_bootstrap.cc:492] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861: No bootstrap required, opened a new log
I20260811 00:17:30.226409 7070 ts_tablet_manager.cc:1397] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.006s
I20260811 00:17:30.226433 7066 tablet_bootstrap.cc:492] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4: No bootstrap required, opened a new log
I20260811 00:17:30.226743 7066 ts_tablet_manager.cc:1397] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 00:17:30.231844 7069 tablet_bootstrap.cc:492] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3: No bootstrap required, opened a new log
I20260811 00:17:30.232117 7069 ts_tablet_manager.cc:1397] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.008s
I20260811 00:17:30.232368 6568 heartbeater.cc:503] Master 127.7.94.62:41787 was elected leader, sending a full tablet report...
I20260811 00:17:30.235242 7070 raft_consensus.cc:348] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaf78c04d38e4dc99ceec0e15d6395c4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36937 } } peers { permanent_uuid: "d1b62abe883a4e2f960e415cc90f5db3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36569 } } peers { permanent_uuid: "f61580d4df6340fc96575786e1864861" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 40785 } }
I20260811 00:17:30.235597 7070 raft_consensus.cc:374] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:30.235697 7070 raft_consensus.cc:729] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f61580d4df6340fc96575786e1864861, State: Initialized, Role: FOLLOWER
I20260811 00:17:30.236363 7070 consensus_queue.cc:260] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaf78c04d38e4dc99ceec0e15d6395c4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36937 } } peers { permanent_uuid: "d1b62abe883a4e2f960e415cc90f5db3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36569 } } peers { permanent_uuid: "f61580d4df6340fc96575786e1864861" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 40785 } }
I20260811 00:17:30.240974 7066 raft_consensus.cc:348] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaf78c04d38e4dc99ceec0e15d6395c4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36937 } } peers { permanent_uuid: "d1b62abe883a4e2f960e415cc90f5db3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36569 } } peers { permanent_uuid: "f61580d4df6340fc96575786e1864861" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 40785 } }
I20260811 00:17:30.241503 7066 raft_consensus.cc:374] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:30.241642 7066 raft_consensus.cc:729] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaf78c04d38e4dc99ceec0e15d6395c4, State: Initialized, Role: FOLLOWER
I20260811 00:17:30.242075 7035 heartbeater.cc:503] Master 127.7.94.62:41787 was elected leader, sending a full tablet report...
I20260811 00:17:30.242444 7066 consensus_queue.cc:260] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaf78c04d38e4dc99ceec0e15d6395c4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36937 } } peers { permanent_uuid: "d1b62abe883a4e2f960e415cc90f5db3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36569 } } peers { permanent_uuid: "f61580d4df6340fc96575786e1864861" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 40785 } }
I20260811 00:17:30.242775 7070 ts_tablet_manager.cc:1428] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861: Time spent starting tablet: real 0.016s user 0.008s sys 0.007s
I20260811 00:17:30.242695 7069 raft_consensus.cc:348] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaf78c04d38e4dc99ceec0e15d6395c4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36937 } } peers { permanent_uuid: "d1b62abe883a4e2f960e415cc90f5db3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36569 } } peers { permanent_uuid: "f61580d4df6340fc96575786e1864861" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 40785 } }
I20260811 00:17:30.243237 7069 raft_consensus.cc:374] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:30.243407 7069 raft_consensus.cc:729] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b62abe883a4e2f960e415cc90f5db3, State: Initialized, Role: FOLLOWER
I20260811 00:17:30.244143 7069 consensus_queue.cc:260] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaf78c04d38e4dc99ceec0e15d6395c4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36937 } } peers { permanent_uuid: "d1b62abe883a4e2f960e415cc90f5db3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36569 } } peers { permanent_uuid: "f61580d4df6340fc96575786e1864861" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 40785 } }
I20260811 00:17:30.244568 7066 ts_tablet_manager.cc:1428] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4: Time spent starting tablet: real 0.018s user 0.015s sys 0.000s
I20260811 00:17:30.245512 6799 heartbeater.cc:503] Master 127.7.94.62:41787 was elected leader, sending a full tablet report...
I20260811 00:17:30.245963 7069 ts_tablet_manager.cc:1428] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260811 00:17:30.320859 7083 raft_consensus.cc:482] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:17:30.321165 7083 raft_consensus.cc:504] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaf78c04d38e4dc99ceec0e15d6395c4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36937 } } peers { permanent_uuid: "d1b62abe883a4e2f960e415cc90f5db3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36569 } } peers { permanent_uuid: "f61580d4df6340fc96575786e1864861" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 40785 } }
I20260811 00:17:30.322971 7083 leader_election.cc:288] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1b62abe883a4e2f960e415cc90f5db3 (127.7.94.2:36569), f61580d4df6340fc96575786e1864861 (127.7.94.3:40785)
I20260811 00:17:30.329977 7065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.323565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:59231 (local address 127.7.94.3:40785)
0811 00:17:30.323798 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:30.323806 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:30.324398 (+ 592us) server_negotiation.cc:407] Connection header received
0811 00:17:30.324675 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:30.324679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:30.324739 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:30.324864 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:30.325554 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.326241 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.328169 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.328920 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.329062 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:30.329176 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:30.329545 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 00:17:30.329714 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 00:17:30.329985 7062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.323409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:54717 (local address 127.7.94.2:36569)
0811 00:17:30.323624 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:30.323634 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:30.324411 (+ 777us) server_negotiation.cc:407] Connection header received
0811 00:17:30.324627 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:30.324631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:30.324703 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:30.324814 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:30.325501 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.326189 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.328227 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.328932 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.329065 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:30.329178 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:30.329554 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 00:17:30.329717 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:17:30.330127 7086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.323384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:36569 (local address 127.7.94.1:54717)
0811 00:17:30.324304 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:30.324319 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:30.324481 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:30.324856 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:30.324860 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:30.324882 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:30.325320 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:30.325335 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.326319 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.326327 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:30.327971 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:30.327982 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.329150 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.329157 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:30.329351 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:30.329402 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 00:17:30.329515 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 00:17:30.329722 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"mutex_wait_us":177,"thread_start_us":619,"threads_started":1}
I20260811 00:17:30.330291 7085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.323224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:40785 (local address 127.7.94.1:59231)
0811 00:17:30.324257 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:30.324284 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:30.324509 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:30.324880 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:30.324884 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:30.324906 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:30.325388 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:30.325399 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.326384 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.326389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:30.327969 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:30.327982 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.329116 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.329122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:30.329341 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:30.329390 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 00:17:30.329511 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 00:17:30.329713 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"mutex_wait_us":76,"spinlock_wait_cycles":51584,"thread_start_us":614,"threads_started":1}
I20260811 00:17:30.331182 6727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60415bbd72da4d57a4d8edbb45de7397" candidate_uuid: "aaf78c04d38e4dc99ceec0e15d6395c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b62abe883a4e2f960e415cc90f5db3" is_pre_election: true
I20260811 00:17:30.331370 6957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60415bbd72da4d57a4d8edbb45de7397" candidate_uuid: "aaf78c04d38e4dc99ceec0e15d6395c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f61580d4df6340fc96575786e1864861" is_pre_election: true
I20260811 00:17:30.331718 6727 raft_consensus.cc:2454] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaf78c04d38e4dc99ceec0e15d6395c4 in term 0.
I20260811 00:17:30.331954 6957 raft_consensus.cc:2454] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaf78c04d38e4dc99ceec0e15d6395c4 in term 0.
I20260811 00:17:30.332448 6356 leader_election.cc:302] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aaf78c04d38e4dc99ceec0e15d6395c4, d1b62abe883a4e2f960e415cc90f5db3; no voters:
I20260811 00:17:30.332844 7083 raft_consensus.cc:2781] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:17:30.333000 7083 raft_consensus.cc:482] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:17:30.333104 7083 raft_consensus.cc:3037] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:30.335304 7083 raft_consensus.cc:504] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaf78c04d38e4dc99ceec0e15d6395c4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36937 } } peers { permanent_uuid: "d1b62abe883a4e2f960e415cc90f5db3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36569 } } peers { permanent_uuid: "f61580d4df6340fc96575786e1864861" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 40785 } }
I20260811 00:17:30.335872 7083 leader_election.cc:288] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [CANDIDATE]: Term 1 election: Requested vote from peers d1b62abe883a4e2f960e415cc90f5db3 (127.7.94.2:36569), f61580d4df6340fc96575786e1864861 (127.7.94.3:40785)
I20260811 00:17:30.336392 6957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60415bbd72da4d57a4d8edbb45de7397" candidate_uuid: "aaf78c04d38e4dc99ceec0e15d6395c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f61580d4df6340fc96575786e1864861"
I20260811 00:17:30.336392 6727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60415bbd72da4d57a4d8edbb45de7397" candidate_uuid: "aaf78c04d38e4dc99ceec0e15d6395c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1b62abe883a4e2f960e415cc90f5db3"
I20260811 00:17:30.336599 6957 raft_consensus.cc:3037] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:30.336599 6727 raft_consensus.cc:3037] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:30.339881 6727 raft_consensus.cc:2454] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaf78c04d38e4dc99ceec0e15d6395c4 in term 1.
I20260811 00:17:30.339881 6957 raft_consensus.cc:2454] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaf78c04d38e4dc99ceec0e15d6395c4 in term 1.
I20260811 00:17:30.340265 6358 leader_election.cc:302] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aaf78c04d38e4dc99ceec0e15d6395c4, f61580d4df6340fc96575786e1864861; no voters:
I20260811 00:17:30.340615 7083 raft_consensus.cc:2781] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:17:30.341082 7083 raft_consensus.cc:686] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [term 1 LEADER]: Becoming Leader. State: Replica: aaf78c04d38e4dc99ceec0e15d6395c4, State: Running, Role: LEADER
I20260811 00:17:30.341619 7083 consensus_queue.cc:237] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaf78c04d38e4dc99ceec0e15d6395c4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36937 } } peers { permanent_uuid: "d1b62abe883a4e2f960e415cc90f5db3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36569 } } peers { permanent_uuid: "f61580d4df6340fc96575786e1864861" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 40785 } }
I20260811 00:17:30.345865 6219 catalog_manager.cc:5495] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to aaf78c04d38e4dc99ceec0e15d6395c4 (127.7.94.1). New cstate: current_term: 1 leader_uuid: "aaf78c04d38e4dc99ceec0e15d6395c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaf78c04d38e4dc99ceec0e15d6395c4" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 36937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1b62abe883a4e2f960e415cc90f5db3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f61580d4df6340fc96575786e1864861" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 40785 } health_report { overall_health: UNKNOWN } } }
W20260811 00:17:30.380753 7036 tablet.cc:2369] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:17:30.388302 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.382494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34108 (local address 127.7.94.1:36937)
0811 00:17:30.382684 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:30.382689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:30.382706 (+ 17us) server_negotiation.cc:407] Connection header received
0811 00:17:30.382853 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:30.382857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:30.382944 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:30.383065 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:17:30.383394 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.384075 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.386105 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.386497 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.386588 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:30.387234 (+ 646us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:17:30.387713 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:17:30.387818 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:30.387925 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 00:17:30.387984 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 00:17:30.396548 6957 raft_consensus.cc:1261] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861 [term 1 FOLLOWER]: Refusing update from remote peer aaf78c04d38e4dc99ceec0e15d6395c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:17:30.396734 6727 raft_consensus.cc:1261] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3 [term 1 FOLLOWER]: Refusing update from remote peer aaf78c04d38e4dc99ceec0e15d6395c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:17:30.397928 7087 consensus_queue.cc:1035] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1b62abe883a4e2f960e415cc90f5db3" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:17:30.398423 7083 consensus_queue.cc:1035] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f61580d4df6340fc96575786e1864861" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 00:17:30.416283 6800 tablet.cc:2369] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:17:30.422955 6224 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:17:30.423305 6224 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:17:30.426877 6224 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be65352ccd4745a6ac817af9c26118a5: Sending DeleteTablet for 3 replicas of tablet 60415bbd72da4d57a4d8edbb45de7397
I20260811 00:17:30.427971 6685 tablet_service.cc:1514] Processing DeleteTablet for tablet 60415bbd72da4d57a4d8edbb45de7397 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:17:30 UTC) from {username='slave'} at 127.0.0.1:56926
I20260811 00:17:30.428025 6444 tablet_service.cc:1514] Processing DeleteTablet for tablet 60415bbd72da4d57a4d8edbb45de7397 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:17:30 UTC) from {username='slave'} at 127.0.0.1:34096
I20260811 00:17:30.428792 7109 tablet_replica.cc:330] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3: stopping tablet replica
I20260811 00:17:30.428856 6923 tablet_service.cc:1514] Processing DeleteTablet for tablet 60415bbd72da4d57a4d8edbb45de7397 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:17:30 UTC) from {username='slave'} at 127.0.0.1:36142
I20260811 00:17:30.429199 7109 raft_consensus.cc:2229] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:17:30.430130 7109 raft_consensus.cc:2258] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:17:30.430356 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 6324
I20260811 00:17:30.432350 7109 ts_tablet_manager.cc:1905] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:17:30.440986 7109 ts_tablet_manager.cc:1918] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:17:30.441282 7109 log.cc:1196] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/60415bbd72da4d57a4d8edbb45de7397
I20260811 00:17:30.441959 7113 tablet_replica.cc:330] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4: stopping tablet replica
I20260811 00:17:30.441967 7109 ts_tablet_manager.cc:1939] T 60415bbd72da4d57a4d8edbb45de7397 P d1b62abe883a4e2f960e415cc90f5db3: Deleting consensus metadata
I20260811 00:17:30.442571 7113 raft_consensus.cc:2229] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:17:30.443301 7113 raft_consensus.cc:2258] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:17:30.444139 6189 catalog_manager.cc:4841] TS d1b62abe883a4e2f960e415cc90f5db3 (127.7.94.2:36569): tablet 60415bbd72da4d57a4d8edbb45de7397 (table test-table [id=ca89f5b2c185466ba235d63f0cebdf55]) successfully deleted
I20260811 00:17:30.444557 7114 tablet_replica.cc:330] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861: stopping tablet replica
I20260811 00:17:30.445032 7114 raft_consensus.cc:2229] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:17:30.445506 7113 ts_tablet_manager.cc:1905] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:17:30.445644 7114 raft_consensus.cc:2258] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:17:30.447850 7114 ts_tablet_manager.cc:1905] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:17:30.451727 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.431330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34112 (local address 127.7.94.1:36937)
0811 00:17:30.431510 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:30.431516 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:30.442717 (+ 11201us) server_negotiation.cc:407] Connection header received
0811 00:17:30.443110 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:30.443116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:30.443227 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:30.443316 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:30.444051 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.445140 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.447092 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.447771 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.447948 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:30.449524 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:30.449536 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:30.449554 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:30.450404 (+ 850us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:30.450465 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:30.450496 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:30.450608 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:30.451162 (+ 554us) server_negotiation.cc:299] Negotiation successful
0811 00:17:30.451271 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 00:17:30.452701 6560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:30.455579 7114 ts_tablet_manager.cc:1918] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:17:30.455735 7114 log.cc:1196] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/60415bbd72da4d57a4d8edbb45de7397
I20260811 00:17:30.456357 7114 ts_tablet_manager.cc:1939] T 60415bbd72da4d57a4d8edbb45de7397 P f61580d4df6340fc96575786e1864861: Deleting consensus metadata
I20260811 00:17:30.458105 6191 catalog_manager.cc:4841] TS f61580d4df6340fc96575786e1864861 (127.7.94.3:40785): tablet 60415bbd72da4d57a4d8edbb45de7397 (table test-table [id=ca89f5b2c185466ba235d63f0cebdf55]) successfully deleted
I20260811 00:17:30.546203 7113 ts_tablet_manager.cc:1918] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:17:30.546787 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 6324
I20260811 00:17:30.546804 7113 log.cc:1196] T 60415bbd72da4d57a4d8edbb45de7397 P aaf78c04d38e4dc99ceec0e15d6395c4: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/60415bbd72da4d57a4d8edbb45de7397
I20260811 00:17:30.573525 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 6577
W20260811 00:17:30.573529 6192 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.94.1:36937 (error 108)
I20260811 00:17:30.575448 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.574965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:36937 (local address 127.0.0.1:34114)
0811 00:17:30.575166 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:30.575287 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:36937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 00:17:30.575654 6192 catalog_manager.cc:4584] TS aaf78c04d38e4dc99ceec0e15d6395c4 (127.7.94.1:36937): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 60415bbd72da4d57a4d8edbb45de7397: Network error: Client connection negotiation failed: client connection to 127.7.94.1:36937: connect: Connection refused (error 111)
I20260811 00:17:30.580698 7062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.574392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56936 (local address 127.7.94.2:36569)
0811 00:17:30.574578 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:30.574585 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:30.574702 (+ 117us) server_negotiation.cc:407] Connection header received
0811 00:17:30.574914 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:30.574933 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:30.575068 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:30.575185 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:30.575652 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.576461 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.577883 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.578262 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.578351 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:30.579383 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:30.579390 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:30.579403 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:30.579981 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:30.580025 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:30.580045 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:30.580127 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:30.580265 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 00:17:30.580350 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 00:17:30.581519 6792 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:30.629325 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.628781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:36937 (local address 127.0.0.1:34118)
0811 00:17:30.628992 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:30.629072 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:36937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 00:17:30.630259 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.629938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:36937 (local address 127.0.0.1:34122)
0811 00:17:30.630075 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:30.630163 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:36937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 00:17:30.670473 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 6577
I20260811 00:17:30.690678 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.690233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:36937 (local address 127.0.0.1:34126)
0811 00:17:30.690427 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:30.690512 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:36937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 00:17:30.691769 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.691411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:36937 (local address 127.0.0.1:34140)
0811 00:17:30.691531 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:30.691605 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:36937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 00:17:30.698956 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 6806
I20260811 00:17:30.705353 7065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.699692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36158 (local address 127.7.94.3:40785)
0811 00:17:30.699860 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:30.699865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:30.699883 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:17:30.700009 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:30.700014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:30.700109 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:30.700202 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:30.700549 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.701334 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.702593 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:30.702960 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:30.703050 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:30.704073 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:30.704081 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:30.704096 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:30.704639 (+ 543us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:30.704682 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:30.704705 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:30.704787 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:30.704900 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 00:17:30.704978 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 00:17:30.705950 7026 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:30.772432 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.771887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:36937 (local address 127.0.0.1:34152)
0811 00:17:30.772124 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:30.772241 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:36937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 00:17:30.773618 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:30.773117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:36937 (local address 127.0.0.1:34156)
0811 00:17:30.773387 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:30.773477 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:36937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
I20260811 00:17:30.792622 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 6806
I20260811 00:17:30.815502 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 6161
I20260811 00:17:30.816557 6276 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:30.930784 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 6161
2026-08-11T00:17:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2812 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:17:31 dist-test-slave-8hhm krb5kdc[7132](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:17:31 dist-test-slave-8hhm krb5kdc[7132](info): set up 2 sockets
Aug 11 00:17:31 dist-test-slave-8hhm krb5kdc[7132](info): commencing operation
krb5kdc: starting...
W20260811 00:17:32.289609 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:17:32 dist-test-slave-8hhm krb5kdc[7132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:17:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:17:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:17:32.593063 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:38297
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:38297
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:17:32.935402 7148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:32.935673 7148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:32.935763 7148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:32.942790 7148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:17:32.942889 7148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:32.942960 7148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:32.943028 7148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:17:32.943066 7148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:17:32.952742 7148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:38297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:38297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:32.953971 7148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:32.955219 7148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:32.966795 7165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:32.966845 7171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:32.966868 7168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:32.967633 7148 server_base.cc:1034] running on GCE node
I20260811 00:17:32.968410 7148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:32.969826 7148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:32.971068 7148 hybrid_clock.cc:648] HybridClock initialized: now 1786407452971031 us; error 31 us; skew 500 ppm
I20260811 00:17:32.971417 7148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:32 dist-test-slave-8hhm krb5kdc[7132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407452, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:32.974690 7148 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:17:32.976009 7148 webserver.cc:460] Webserver started at http://127.7.94.62:40841/ using document root <none> and password file <none>
I20260811 00:17:32.976627 7148 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:32.976719 7148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:32.976962 7148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:32.979804 7148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "b1eb21f3761e44729a5d9338e176479f"
format_stamp: "Formatted at 2026-08-11 00:17:32 on dist-test-slave-8hhm"
server_key: "5cb8afca4d062f940001bc79a6e3886b"
server_key_iv: "e80a2903c708a342463330a82a787e25"
server_key_version: "encryptionkey@0"
I20260811 00:17:32.980486 7148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "b1eb21f3761e44729a5d9338e176479f"
format_stamp: "Formatted at 2026-08-11 00:17:32 on dist-test-slave-8hhm"
server_key: "5cb8afca4d062f940001bc79a6e3886b"
server_key_iv: "e80a2903c708a342463330a82a787e25"
server_key_version: "encryptionkey@0"
I20260811 00:17:32.985450 7148 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260811 00:17:32.988683 7185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:32.989949 7148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:17:32.990130 7148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "b1eb21f3761e44729a5d9338e176479f"
format_stamp: "Formatted at 2026-08-11 00:17:32 on dist-test-slave-8hhm"
server_key: "5cb8afca4d062f940001bc79a6e3886b"
server_key_iv: "e80a2903c708a342463330a82a787e25"
server_key_version: "encryptionkey@0"
I20260811 00:17:32.990339 7148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:33.014863 7148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:33.018693 7148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:33.019014 7148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:33.037787 7148 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:38297
I20260811 00:17:33.037820 7258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:38297 every 8 connection(s)
I20260811 00:17:33.039096 7148 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:17:33.042770 7259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:33.043057 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 7148
I20260811 00:17:33.043244 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:17:33.043673 7544 external_mini_cluster.cc:1428] Setting key 769285e0672c05be2a2b96538cc9a241
I20260811 00:17:33.056254 7259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f: Bootstrap starting.
Aug 11 00:17:33 dist-test-slave-8hhm krb5kdc[7132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:17:33.059495 7259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:33.060698 7259 log.cc:824] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:33.064214 7259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f: No bootstrap required, opened a new log
I20260811 00:17:33.068715 7266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:33.045478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49598 (local address 127.7.94.62:38297)
0811 00:17:33.046276 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:33.046286 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:33.046324 (+ 38us) server_negotiation.cc:407] Connection header received
0811 00:17:33.048628 (+ 2304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:33.048669 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:33.049170 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:33.049676 (+ 506us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:33.050976 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:33.053032 (+ 2056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:33.054495 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:33.055511 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:33.055703 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:33.060213 (+ 4510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:33.060223 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:33.060241 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:33.060291 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:33.064151 (+ 3860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:33.064902 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:33.064910 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:33.064921 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:33.065148 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:33.065552 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:33.065557 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:33.065560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:33.066233 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:33.066531 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:33.066940 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 00:17:33.067391 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":388,"threads_started":1}
I20260811 00:17:33.073734 7259 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1eb21f3761e44729a5d9338e176479f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 38297 } }
I20260811 00:17:33.074064 7259 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:33.074131 7259 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1eb21f3761e44729a5d9338e176479f, State: Initialized, Role: FOLLOWER
I20260811 00:17:33.074648 7259 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1eb21f3761e44729a5d9338e176479f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 38297 } }
I20260811 00:17:33.074893 7259 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:17:33.075034 7259 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:17:33.075227 7259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:33.077554 7259 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1eb21f3761e44729a5d9338e176479f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 38297 } }
I20260811 00:17:33.078068 7259 leader_election.cc:302] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1eb21f3761e44729a5d9338e176479f; no voters:
I20260811 00:17:33.078626 7259 leader_election.cc:288] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:17:33.078775 7270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:17:33.080178 7270 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f [term 1 LEADER]: Becoming Leader. State: Replica: b1eb21f3761e44729a5d9338e176479f, State: Running, Role: LEADER
I20260811 00:17:33.080677 7259 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:17:33.080669 7270 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1eb21f3761e44729a5d9338e176479f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 38297 } }
I20260811 00:17:33.083570 7272 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1eb21f3761e44729a5d9338e176479f. Latest consensus state: current_term: 1 leader_uuid: "b1eb21f3761e44729a5d9338e176479f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1eb21f3761e44729a5d9338e176479f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 38297 } } }
I20260811 00:17:33.083925 7272 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:33.083689 7271 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1eb21f3761e44729a5d9338e176479f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1eb21f3761e44729a5d9338e176479f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 38297 } } }
I20260811 00:17:33.084069 7271 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:33.084807 7290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:17:33.090994 7290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:17:33.102118 7290 catalog_manager.cc:1347] Generated new cluster ID: 1f17d55dc182496c9549520e66045299
I20260811 00:17:33.102246 7290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:17:33.116828 7290 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:17:33.118069 7290 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:17:33.125512 7290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f: Generated new TSK 0
I20260811 00:17:33.126212 7290 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
I20260811 00:17:33.214404 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:38297
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:17:33.579576 7310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:33.579855 7310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:33.580034 7310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:33.587535 7310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:33.587708 7310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:33.587874 7310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:17:33.597254 7310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:38297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:33.598477 7310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:33.599768 7310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:33.612914 7317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:33.612859 7316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:33.612954 7320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:33.613855 7310 server_base.cc:1034] running on GCE node
I20260811 00:17:33.614439 7310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:33.615348 7310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:33.616604 7310 hybrid_clock.cc:648] HybridClock initialized: now 1786407453616559 us; error 52 us; skew 500 ppm
I20260811 00:17:33.616930 7310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:33 dist-test-slave-8hhm krb5kdc[7132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407453, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:33.623546 7310 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:17:33.625138 7310 webserver.cc:460] Webserver started at http://127.7.94.1:39375/ using document root <none> and password file <none>
I20260811 00:17:33.625965 7310 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:33.626114 7310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:33.626478 7310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:33.629422 7310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "4dbe85c72a8b450bb1c4eb0a8e200804"
format_stamp: "Formatted at 2026-08-11 00:17:33 on dist-test-slave-8hhm"
server_key: "1222e40b9577de21ffa3572105ef8f0a"
server_key_iv: "ffd66034b2175aa726db02e6b9e6d68a"
server_key_version: "encryptionkey@0"
I20260811 00:17:33.630194 7310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "4dbe85c72a8b450bb1c4eb0a8e200804"
format_stamp: "Formatted at 2026-08-11 00:17:33 on dist-test-slave-8hhm"
server_key: "1222e40b9577de21ffa3572105ef8f0a"
server_key_iv: "ffd66034b2175aa726db02e6b9e6d68a"
server_key_version: "encryptionkey@0"
I20260811 00:17:33.635059 7310 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 00:17:33.638239 7335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:33.639416 7310 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 00:17:33.639582 7310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "4dbe85c72a8b450bb1c4eb0a8e200804"
format_stamp: "Formatted at 2026-08-11 00:17:33 on dist-test-slave-8hhm"
server_key: "1222e40b9577de21ffa3572105ef8f0a"
server_key_iv: "ffd66034b2175aa726db02e6b9e6d68a"
server_key_version: "encryptionkey@0"
I20260811 00:17:33.639780 7310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:33.662981 7310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:33.666800 7310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:33.667088 7310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:33.668002 7310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:33.669350 7310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:17:33.669422 7310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:33.669588 7310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:33.669648 7310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:33.702726 7310 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:38655
I20260811 00:17:33.702809 7558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:38655 every 8 connection(s)
I20260811 00:17:33.703984 7310 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:17:33.707746 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 7310
I20260811 00:17:33.707931 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:17:33.708520 7544 external_mini_cluster.cc:1428] Setting key 3808ce21bf5df40bd5897d0b2fc5a520
Aug 11 00:17:33 dist-test-slave-8hhm krb5kdc[7132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407453, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:17:33.724720 7562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:33.706489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:58623 (local address 127.7.94.62:38297)
0811 00:17:33.706891 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:33.706896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:33.709134 (+ 2238us) server_negotiation.cc:407] Connection header received
0811 00:17:33.710217 (+ 1083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:33.710223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:33.710328 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:33.710439 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:33.711886 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:33.712591 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:33.714242 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:33.714658 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:33.714791 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:33.718644 (+ 3853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:33.718649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:33.718652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:33.718681 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:33.721403 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:33.722299 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:33.722303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:33.722305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:33.722423 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:33.722899 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:33.722902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:33.722904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:33.723209 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:33.723370 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:33.724007 (+ 637us) server_negotiation.cc:299] Negotiation successful
0811 00:17:33.724261 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":192,"threads_started":1}
I20260811 00:17:33.725301 7563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:33.707018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:38297 (local address 127.7.94.1:58623)
0811 00:17:33.707907 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:33.707959 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:33.709973 (+ 2014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:33.710665 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:33.710672 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:33.711152 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:17:33.711690 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:33.711718 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:33.712746 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:33.712751 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:33.714088 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:33.714101 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:33.714878 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:33.714887 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:33.715124 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:33.715815 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:17:33.715837 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:17:33.718436 (+ 2599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:17:33.721529 (+ 3093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:17:33.721540 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:17:33.721555 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:17:33.722167 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:17:33.722527 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:17:33.722535 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:17:33.722541 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:17:33.722787 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:17:33.723349 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:17:33.723355 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:17:33.723637 (+ 282us) client_negotiation.cc:769] Sending connection context
0811 00:17:33.723846 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 00:17:33.724198 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":337,"threads_started":1}
I20260811 00:17:33.726787 7559 heartbeater.cc:347] Connected to a master server at 127.7.94.62:38297
I20260811 00:17:33.727151 7559 heartbeater.cc:464] Registering TS with master...
I20260811 00:17:33.727972 7559 heartbeater.cc:511] Master 127.7.94.62:38297 requested a full tablet report, sending...
I20260811 00:17:33.730186 7206 ts_manager.cc:194] Registered new tserver with Master: 4dbe85c72a8b450bb1c4eb0a8e200804 (127.7.94.1:38655)
I20260811 00:17:33.731536 7206 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:58623
WARNING: no policy specified for kudu/127.7.94.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
I20260811 00:17:33.775868 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:38297
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:17:34.108513 7571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:34.108800 7571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:34.108897 7571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:34.116014 7571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:34.116123 7571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:34.116322 7571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:17:34.125538 7571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:38297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:34.126797 7571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:34.128175 7571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:34.141064 7591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:34.141110 7579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:34.141528 7578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:34.141839 7571 server_base.cc:1034] running on GCE node
I20260811 00:17:34.142441 7571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:34.143357 7571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:34.144591 7571 hybrid_clock.cc:648] HybridClock initialized: now 1786407454144520 us; error 76 us; skew 500 ppm
I20260811 00:17:34.144902 7571 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:34 dist-test-slave-8hhm krb5kdc[7132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407454, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:34.148181 7571 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:17:34.149631 7571 webserver.cc:460] Webserver started at http://127.7.94.2:43383/ using document root <none> and password file <none>
I20260811 00:17:34.150328 7571 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:34.150475 7571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:34.150800 7571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:34.153990 7571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "41baca490ebf449b8fd8690d3396ba1c"
format_stamp: "Formatted at 2026-08-11 00:17:34 on dist-test-slave-8hhm"
server_key: "d484565d37ace7e8b2859ba1a8287351"
server_key_iv: "4aa3e40e0da4f48b8b6b4393bd0fefac"
server_key_version: "encryptionkey@0"
I20260811 00:17:34.155170 7571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "41baca490ebf449b8fd8690d3396ba1c"
format_stamp: "Formatted at 2026-08-11 00:17:34 on dist-test-slave-8hhm"
server_key: "d484565d37ace7e8b2859ba1a8287351"
server_key_iv: "4aa3e40e0da4f48b8b6b4393bd0fefac"
server_key_version: "encryptionkey@0"
I20260811 00:17:34.160238 7571 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:17:34.163614 7606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:34.164904 7571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 00:17:34.165086 7571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "41baca490ebf449b8fd8690d3396ba1c"
format_stamp: "Formatted at 2026-08-11 00:17:34 on dist-test-slave-8hhm"
server_key: "d484565d37ace7e8b2859ba1a8287351"
server_key_iv: "4aa3e40e0da4f48b8b6b4393bd0fefac"
server_key_version: "encryptionkey@0"
I20260811 00:17:34.165302 7571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:34.185731 7571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:34.189934 7571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:34.190222 7571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:34.191138 7571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:34.192427 7571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:17:34.192538 7571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:34.192687 7571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:34.192759 7571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:34.225591 7571 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:40859
I20260811 00:17:34.225634 7805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:40859 every 8 connection(s)
I20260811 00:17:34.226892 7571 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:17:34.235176 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 7571
I20260811 00:17:34.235365 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:17:34.235852 7544 external_mini_cluster.cc:1428] Setting key feae7c771d86cdc298afb18b8202597b
Aug 11 00:17:34 dist-test-slave-8hhm krb5kdc[7132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407454, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:17:34.248920 7810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:34.229586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:35727 (local address 127.7.94.62:38297)
0811 00:17:34.230066 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:34.230071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:34.230933 (+ 862us) server_negotiation.cc:407] Connection header received
0811 00:17:34.232774 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:34.232779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:34.232876 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:34.232995 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:34.234477 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.235516 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.237036 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.237750 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.237878 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:34.242009 (+ 4131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:34.242018 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:34.242025 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:34.242079 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:34.245791 (+ 3712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:34.246497 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:34.246505 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:34.246511 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:34.246686 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:34.247096 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:34.247104 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:34.247110 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:34.247476 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:34.247635 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:34.248217 (+ 582us) server_negotiation.cc:299] Negotiation successful
0811 00:17:34.248459 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":282,"threads_started":1}
I20260811 00:17:34.249539 7812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:34.230089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:38297 (local address 127.7.94.2:35727)
0811 00:17:34.230783 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:34.230821 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:34.232512 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:34.233278 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:34.233285 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:34.233773 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:17:34.234298 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:34.234319 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.235665 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.235669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:34.236899 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:34.236912 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.237983 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.237991 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:34.238296 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:34.239039 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:17:34.239081 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:17:34.241767 (+ 2686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:17:34.245918 (+ 4151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:17:34.245924 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:17:34.245935 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:17:34.246374 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:17:34.246797 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:17:34.246800 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:17:34.246802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:17:34.246976 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:17:34.247600 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:17:34.247605 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:17:34.247891 (+ 286us) client_negotiation.cc:769] Sending connection context
0811 00:17:34.248092 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 00:17:34.248451 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":270,"threads_started":1}
I20260811 00:17:34.250970 7807 heartbeater.cc:347] Connected to a master server at 127.7.94.62:38297
I20260811 00:17:34.251331 7807 heartbeater.cc:464] Registering TS with master...
I20260811 00:17:34.252031 7807 heartbeater.cc:511] Master 127.7.94.62:38297 requested a full tablet report, sending...
I20260811 00:17:34.253731 7206 ts_manager.cc:194] Registered new tserver with Master: 41baca490ebf449b8fd8690d3396ba1c (127.7.94.2:40859)
I20260811 00:17:34.254483 7206 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:35727
WARNING: no policy specified for kudu/127.7.94.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
I20260811 00:17:34.306922 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:38297
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:17:34.640196 7816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:34.640491 7816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:34.640645 7816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:34.647773 7816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:34.647903 7816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:34.648065 7816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:17:34.657621 7816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:38297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:34.658840 7816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:34.660102 7816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:34.673440 7828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:34.673463 7826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:34.674146 7825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:34.674187 7816 server_base.cc:1034] running on GCE node
I20260811 00:17:34.674854 7816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:34.675760 7816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:34.676983 7816 hybrid_clock.cc:648] HybridClock initialized: now 1786407454676944 us; error 46 us; skew 500 ppm
I20260811 00:17:34.677347 7816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:34 dist-test-slave-8hhm krb5kdc[7132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407454, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:34.680605 7816 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:17:34.682134 7816 webserver.cc:460] Webserver started at http://127.7.94.3:34531/ using document root <none> and password file <none>
I20260811 00:17:34.682814 7816 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:34.682943 7816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:34.683256 7816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:34.686146 7816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "37595f65bb5f44798f58cbff8f57a342"
format_stamp: "Formatted at 2026-08-11 00:17:34 on dist-test-slave-8hhm"
server_key: "9c2853e1bc29fb22bfdc5b200c6a434c"
server_key_iv: "d466ef3fbdfc60634db4bd9e30b5dfed"
server_key_version: "encryptionkey@0"
I20260811 00:17:34.686934 7816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "37595f65bb5f44798f58cbff8f57a342"
format_stamp: "Formatted at 2026-08-11 00:17:34 on dist-test-slave-8hhm"
server_key: "9c2853e1bc29fb22bfdc5b200c6a434c"
server_key_iv: "d466ef3fbdfc60634db4bd9e30b5dfed"
server_key_version: "encryptionkey@0"
I20260811 00:17:34.691912 7816 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:17:34.695253 7840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:34.696547 7816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:17:34.696718 7816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "37595f65bb5f44798f58cbff8f57a342"
format_stamp: "Formatted at 2026-08-11 00:17:34 on dist-test-slave-8hhm"
server_key: "9c2853e1bc29fb22bfdc5b200c6a434c"
server_key_iv: "d466ef3fbdfc60634db4bd9e30b5dfed"
server_key_version: "encryptionkey@0"
I20260811 00:17:34.696908 7816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:34.713965 7816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:34.718215 7816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:34.718523 7816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:34.719491 7816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:34.720774 7816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:17:34.720846 7816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:34.721031 7816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:34.721086 7816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:34.735037 7559 heartbeater.cc:503] Master 127.7.94.62:38297 was elected leader, sending a full tablet report...
I20260811 00:17:34.756052 7816 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:33695
I20260811 00:17:34.756089 8045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:33695 every 8 connection(s)
I20260811 00:17:34.757282 7816 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:17:34.766899 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 7816
I20260811 00:17:34.767102 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:17:34.767614 7544 external_mini_cluster.cc:1428] Setting key b60279cb9603d10895f6710a26406966
Aug 11 00:17:34 dist-test-slave-8hhm krb5kdc[7132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407454, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:17:34.777007 8048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:34.759837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:53459 (local address 127.7.94.62:38297)
0811 00:17:34.760309 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:34.760313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:34.761232 (+ 919us) server_negotiation.cc:407] Connection header received
0811 00:17:34.763060 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:34.763066 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:34.763164 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:34.763284 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:34.764907 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.765731 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.767329 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.767815 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.767947 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:34.771819 (+ 3872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:34.771823 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:34.771826 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:34.771857 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:34.774417 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:34.775071 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:34.775074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:34.775076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:34.775185 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:34.775588 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:34.775592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:34.775593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:34.775849 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:34.775959 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:34.776574 (+ 615us) server_negotiation.cc:299] Negotiation successful
0811 00:17:34.776749 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":227,"threads_started":1}
I20260811 00:17:34.778280 8049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:34.760353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:38297 (local address 127.7.94.3:53459)
0811 00:17:34.761014 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:34.761088 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:34.762791 (+ 1703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:34.763535 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:34.763557 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:34.764051 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:17:34.764744 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:34.764761 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.765881 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.765885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:34.767179 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:34.767199 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.768043 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.768051 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:34.768295 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:34.769089 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:17:34.769110 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:17:34.771625 (+ 2515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:17:34.774545 (+ 2920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:17:34.774552 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:17:34.774563 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:17:34.774974 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:17:34.775276 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:17:34.775279 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:17:34.775281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:17:34.775490 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:17:34.775955 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:17:34.775960 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:17:34.776260 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 00:17:34.776492 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 00:17:34.776880 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":258,"threads_started":1}
I20260811 00:17:34.779673 8046 heartbeater.cc:347] Connected to a master server at 127.7.94.62:38297
I20260811 00:17:34.779960 8046 heartbeater.cc:464] Registering TS with master...
I20260811 00:17:34.780704 8046 heartbeater.cc:511] Master 127.7.94.62:38297 requested a full tablet report, sending...
I20260811 00:17:34.782315 7206 ts_manager.cc:194] Registered new tserver with Master: 37595f65bb5f44798f58cbff8f57a342 (127.7.94.3:33695)
I20260811 00:17:34.783308 7206 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:53459
I20260811 00:17:34.783332 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:17:34.801122 8048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:34.787526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49604 (local address 127.7.94.62:38297)
0811 00:17:34.787722 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:34.787728 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:34.788250 (+ 522us) server_negotiation.cc:407] Connection header received
0811 00:17:34.788435 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:34.788441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:34.788512 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:34.788609 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:34.790071 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.790754 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.792093 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.792491 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.792626 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:34.795852 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:34.795858 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:34.795863 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:34.795908 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:34.798347 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:34.799130 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:34.799135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:34.799138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:34.799263 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:34.799736 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:34.799741 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:34.799744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:34.800113 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:34.800363 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:34.800696 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 00:17:34.800837 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 00:17:34.804765 7206 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:17:34.811673 7206 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:17:34.851359 8073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:34.825675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44824 (local address 127.7.94.1:38655)
0811 00:17:34.832079 (+ 6404us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:34.832091 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:34.842734 (+ 10643us) server_negotiation.cc:407] Connection header received
0811 00:17:34.843079 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:34.843087 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:34.843361 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:34.843598 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:34.844514 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.845644 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.849373 (+ 3729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.850280 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.850429 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:34.850672 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:34.850912 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 00:17:34.851053 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6209,"thread_start_us":6038,"threads_started":1}
I20260811 00:17:34.851357 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:34.823381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49180 (local address 127.7.94.2:40859)
0811 00:17:34.842924 (+ 19543us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:34.842932 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:34.842960 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:17:34.843054 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:34.843061 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:34.843300 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:34.843487 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:34.844328 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.845984 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.848678 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.849875 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.849997 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:34.850197 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:34.850814 (+ 617us) server_negotiation.cc:299] Negotiation successful
0811 00:17:34.850992 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19389,"thread_start_us":19268,"threads_started":1}
I20260811 00:17:34.851395 8064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:34.823107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:40859 (local address 127.0.0.1:49180)
0811 00:17:34.841243 (+ 18136us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:34.841302 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:34.841496 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:34.843497 (+ 2001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:34.843501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:34.843531 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:34.844117 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:34.844133 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.846100 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.846108 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:34.848514 (+ 2406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:34.848527 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.850318 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.850324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:34.850566 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:34.850637 (+ 71us) client_negotiation.cc:769] Sending connection context
0811 00:17:34.850761 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 00:17:34.850947 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17824,"mutex_wait_us":513,"thread_start_us":19159,"threads_started":1}
I20260811 00:17:34.852275 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:34.824843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37150 (local address 127.7.94.3:33695)
0811 00:17:34.841631 (+ 16788us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:34.841641 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:34.842575 (+ 934us) server_negotiation.cc:407] Connection header received
0811 00:17:34.842708 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:34.842716 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:34.842984 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:34.843192 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:34.843979 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.845770 (+ 1791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.849319 (+ 3549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.850642 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.850770 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:34.850970 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:34.851890 (+ 920us) server_negotiation.cc:299] Negotiation successful
0811 00:17:34.852004 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16599,"thread_start_us":16426,"threads_started":1}
I20260811 00:17:34.852298 8061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:34.823999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38655 (local address 127.0.0.1:44824)
0811 00:17:34.842598 (+ 18599us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:34.842619 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:34.842843 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:34.843853 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:34.843858 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:34.843881 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:34.844360 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:34.844381 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.845796 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.845802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:34.849147 (+ 3345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:34.849162 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.850485 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.850491 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:34.850709 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:34.850747 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 00:17:34.851793 (+ 1046us) client_negotiation.cc:240] Negotiation successful
0811 00:17:34.851920 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18460,"mutex_wait_us":260,"thread_start_us":17824,"threads_started":1}
I20260811 00:17:34.852837 8068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:34.823380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:33695 (local address 127.0.0.1:37150)
0811 00:17:34.842218 (+ 18838us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:34.842243 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:34.842381 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:34.843207 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:34.843214 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:34.843244 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:34.843767 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:34.843785 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.845906 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.845910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:34.849147 (+ 3237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:34.849162 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.851123 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.851130 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:34.851300 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:34.851341 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 00:17:34.851440 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 00:17:34.852582 (+ 1142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18643,"mutex_wait_us":419,"thread_start_us":16577,"threads_started":1}
I20260811 00:17:34.855891 7698 tablet_service.cc:1467] Processing CreateTablet for tablet 816031580b114b1b94319a5e1a18cc70 (DEFAULT_TABLE table=test-table [id=7e41ba9e8892445baa2c84c840d08744]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:17:34.856243 7431 tablet_service.cc:1467] Processing CreateTablet for tablet 816031580b114b1b94319a5e1a18cc70 (DEFAULT_TABLE table=test-table [id=7e41ba9e8892445baa2c84c840d08744]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:17:34.856243 7919 tablet_service.cc:1467] Processing CreateTablet for tablet 816031580b114b1b94319a5e1a18cc70 (DEFAULT_TABLE table=test-table [id=7e41ba9e8892445baa2c84c840d08744]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:17:34.857391 7698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 816031580b114b1b94319a5e1a18cc70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:34.857692 7919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 816031580b114b1b94319a5e1a18cc70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:34.857710 7431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 816031580b114b1b94319a5e1a18cc70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:34.868541 8074 tablet_bootstrap.cc:492] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c: Bootstrap starting.
I20260811 00:17:34.871759 8074 tablet_bootstrap.cc:654] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:34.872778 8074 log.cc:824] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:34.874104 8080 tablet_bootstrap.cc:492] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804: Bootstrap starting.
I20260811 00:17:34.875070 8078 tablet_bootstrap.cc:492] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342: Bootstrap starting.
I20260811 00:17:34.879673 8078 tablet_bootstrap.cc:654] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:34.881291 8078 log.cc:824] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:34.888144 8074 tablet_bootstrap.cc:492] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c: No bootstrap required, opened a new log
I20260811 00:17:34.888195 8080 tablet_bootstrap.cc:654] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:34.888543 8074 ts_tablet_manager.cc:1397] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c: Time spent bootstrapping tablet: real 0.020s user 0.013s sys 0.004s
I20260811 00:17:34.890101 8080 log.cc:824] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:34.893527 8078 tablet_bootstrap.cc:492] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342: No bootstrap required, opened a new log
I20260811 00:17:34.893816 8078 ts_tablet_manager.cc:1397] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342: Time spent bootstrapping tablet: real 0.019s user 0.014s sys 0.004s
I20260811 00:17:34.895180 8080 tablet_bootstrap.cc:492] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804: No bootstrap required, opened a new log
I20260811 00:17:34.895439 8080 ts_tablet_manager.cc:1397] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804: Time spent bootstrapping tablet: real 0.022s user 0.008s sys 0.000s
I20260811 00:17:34.899811 8074 raft_consensus.cc:348] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41baca490ebf449b8fd8690d3396ba1c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40859 } } peers { permanent_uuid: "37595f65bb5f44798f58cbff8f57a342" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33695 } } peers { permanent_uuid: "4dbe85c72a8b450bb1c4eb0a8e200804" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38655 } }
I20260811 00:17:34.900257 8074 raft_consensus.cc:374] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:34.900416 8074 raft_consensus.cc:729] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41baca490ebf449b8fd8690d3396ba1c, State: Initialized, Role: FOLLOWER
I20260811 00:17:34.901170 8074 consensus_queue.cc:260] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41baca490ebf449b8fd8690d3396ba1c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40859 } } peers { permanent_uuid: "37595f65bb5f44798f58cbff8f57a342" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33695 } } peers { permanent_uuid: "4dbe85c72a8b450bb1c4eb0a8e200804" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38655 } }
I20260811 00:17:34.902321 7807 heartbeater.cc:503] Master 127.7.94.62:38297 was elected leader, sending a full tablet report...
I20260811 00:17:34.903076 8074 ts_tablet_manager.cc:1428] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260811 00:17:34.903111 8080 raft_consensus.cc:348] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41baca490ebf449b8fd8690d3396ba1c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40859 } } peers { permanent_uuid: "37595f65bb5f44798f58cbff8f57a342" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33695 } } peers { permanent_uuid: "4dbe85c72a8b450bb1c4eb0a8e200804" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38655 } }
I20260811 00:17:34.903517 8080 raft_consensus.cc:374] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:34.903640 8080 raft_consensus.cc:729] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dbe85c72a8b450bb1c4eb0a8e200804, State: Initialized, Role: FOLLOWER
I20260811 00:17:34.904263 8080 consensus_queue.cc:260] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41baca490ebf449b8fd8690d3396ba1c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40859 } } peers { permanent_uuid: "37595f65bb5f44798f58cbff8f57a342" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33695 } } peers { permanent_uuid: "4dbe85c72a8b450bb1c4eb0a8e200804" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38655 } }
I20260811 00:17:34.905405 8078 raft_consensus.cc:348] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41baca490ebf449b8fd8690d3396ba1c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40859 } } peers { permanent_uuid: "37595f65bb5f44798f58cbff8f57a342" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33695 } } peers { permanent_uuid: "4dbe85c72a8b450bb1c4eb0a8e200804" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38655 } }
I20260811 00:17:34.905884 8078 raft_consensus.cc:374] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:34.906015 8080 ts_tablet_manager.cc:1428] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804: Time spent starting tablet: real 0.010s user 0.002s sys 0.010s
I20260811 00:17:34.906015 8078 raft_consensus.cc:729] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37595f65bb5f44798f58cbff8f57a342, State: Initialized, Role: FOLLOWER
I20260811 00:17:34.906577 8078 consensus_queue.cc:260] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41baca490ebf449b8fd8690d3396ba1c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40859 } } peers { permanent_uuid: "37595f65bb5f44798f58cbff8f57a342" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33695 } } peers { permanent_uuid: "4dbe85c72a8b450bb1c4eb0a8e200804" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38655 } }
I20260811 00:17:34.908970 8046 heartbeater.cc:503] Master 127.7.94.62:38297 was elected leader, sending a full tablet report...
I20260811 00:17:34.909603 8078 ts_tablet_manager.cc:1428] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260811 00:17:34.914358 8090 raft_consensus.cc:482] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:17:34.914592 8090 raft_consensus.cc:504] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41baca490ebf449b8fd8690d3396ba1c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40859 } } peers { permanent_uuid: "37595f65bb5f44798f58cbff8f57a342" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33695 } } peers { permanent_uuid: "4dbe85c72a8b450bb1c4eb0a8e200804" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38655 } }
I20260811 00:17:34.916074 8090 leader_election.cc:288] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41baca490ebf449b8fd8690d3396ba1c (127.7.94.2:40859), 4dbe85c72a8b450bb1c4eb0a8e200804 (127.7.94.1:38655)
I20260811 00:17:34.921527 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:34.916380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:44955 (local address 127.7.94.2:40859)
0811 00:17:34.916590 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:34.916597 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:34.916620 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:17:34.916813 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:34.916817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:34.916869 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:34.916961 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:34.917397 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.918148 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.919812 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.920589 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.920738 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:34.920831 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:34.921107 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 00:17:34.921270 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 00:17:34.921535 8049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:34.916192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:40859 (local address 127.7.94.3:44955)
0811 00:17:34.916507 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:34.916528 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:34.916632 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:34.916987 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:34.916991 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:34.917015 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:34.917261 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:34.917272 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.918291 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.918297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:34.919655 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:34.919667 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.920748 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.920754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:34.920946 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:34.920970 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 00:17:34.921077 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 00:17:34.921208 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":49}
I20260811 00:17:34.922225 8073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:34.916551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:53273 (local address 127.7.94.1:38655)
0811 00:17:34.916747 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:34.916753 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:34.917191 (+ 438us) server_negotiation.cc:407] Connection header received
0811 00:17:34.917400 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:34.917403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:34.917453 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:34.917561 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:34.918085 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.918767 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.920465 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.921308 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.921447 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:34.921547 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:34.921873 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 00:17:34.922008 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 00:17:34.922276 8092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:34.916313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38655 (local address 127.7.94.3:53273)
0811 00:17:34.917085 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:34.917105 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:34.917261 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:34.917577 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:34.917581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:34.917604 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:34.917939 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:34.917947 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.918912 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.918918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:34.920313 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:34.920324 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.921516 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.921526 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:34.921721 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:34.921749 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 00:17:34.921833 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 00:17:34.921935 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":2,"thread_start_us":480,"threads_started":1}
I20260811 00:17:34.922652 7731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "816031580b114b1b94319a5e1a18cc70" candidate_uuid: "37595f65bb5f44798f58cbff8f57a342" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41baca490ebf449b8fd8690d3396ba1c" is_pre_election: true
I20260811 00:17:34.923125 7731 raft_consensus.cc:2454] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37595f65bb5f44798f58cbff8f57a342 in term 0.
I20260811 00:17:34.923240 7470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "816031580b114b1b94319a5e1a18cc70" candidate_uuid: "37595f65bb5f44798f58cbff8f57a342" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dbe85c72a8b450bb1c4eb0a8e200804" is_pre_election: true
I20260811 00:17:34.923738 7470 raft_consensus.cc:2454] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37595f65bb5f44798f58cbff8f57a342 in term 0.
I20260811 00:17:34.923832 7841 leader_election.cc:302] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37595f65bb5f44798f58cbff8f57a342, 41baca490ebf449b8fd8690d3396ba1c; no voters:
I20260811 00:17:34.924244 8090 raft_consensus.cc:2781] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:17:34.924378 8090 raft_consensus.cc:482] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:17:34.924500 8090 raft_consensus.cc:3037] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:34.926889 8090 raft_consensus.cc:504] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41baca490ebf449b8fd8690d3396ba1c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40859 } } peers { permanent_uuid: "37595f65bb5f44798f58cbff8f57a342" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33695 } } peers { permanent_uuid: "4dbe85c72a8b450bb1c4eb0a8e200804" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38655 } }
I20260811 00:17:34.927620 8090 leader_election.cc:288] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [CANDIDATE]: Term 1 election: Requested vote from peers 41baca490ebf449b8fd8690d3396ba1c (127.7.94.2:40859), 4dbe85c72a8b450bb1c4eb0a8e200804 (127.7.94.1:38655)
I20260811 00:17:34.927949 7731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "816031580b114b1b94319a5e1a18cc70" candidate_uuid: "37595f65bb5f44798f58cbff8f57a342" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41baca490ebf449b8fd8690d3396ba1c"
I20260811 00:17:34.928174 7731 raft_consensus.cc:3037] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:34.928242 7470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "816031580b114b1b94319a5e1a18cc70" candidate_uuid: "37595f65bb5f44798f58cbff8f57a342" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dbe85c72a8b450bb1c4eb0a8e200804"
I20260811 00:17:34.928478 7470 raft_consensus.cc:3037] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:34.930591 7731 raft_consensus.cc:2454] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37595f65bb5f44798f58cbff8f57a342 in term 1.
I20260811 00:17:34.930951 7470 raft_consensus.cc:2454] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37595f65bb5f44798f58cbff8f57a342 in term 1.
I20260811 00:17:34.930958 7841 leader_election.cc:302] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37595f65bb5f44798f58cbff8f57a342, 41baca490ebf449b8fd8690d3396ba1c; no voters:
I20260811 00:17:34.931246 8090 raft_consensus.cc:2781] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:17:34.931708 8090 raft_consensus.cc:686] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [term 1 LEADER]: Becoming Leader. State: Replica: 37595f65bb5f44798f58cbff8f57a342, State: Running, Role: LEADER
I20260811 00:17:34.932207 8090 consensus_queue.cc:237] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41baca490ebf449b8fd8690d3396ba1c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40859 } } peers { permanent_uuid: "37595f65bb5f44798f58cbff8f57a342" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33695 } } peers { permanent_uuid: "4dbe85c72a8b450bb1c4eb0a8e200804" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38655 } }
I20260811 00:17:34.937029 7205 catalog_manager.cc:5495] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 reported cstate change: term changed from 0 to 1, leader changed from <none> to 37595f65bb5f44798f58cbff8f57a342 (127.7.94.3). New cstate: current_term: 1 leader_uuid: "37595f65bb5f44798f58cbff8f57a342" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41baca490ebf449b8fd8690d3396ba1c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37595f65bb5f44798f58cbff8f57a342" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4dbe85c72a8b450bb1c4eb0a8e200804" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38655 } health_report { overall_health: UNKNOWN } } }
W20260811 00:17:34.956379 7561 tablet.cc:2369] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:17:34.964239 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:34.958526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37162 (local address 127.7.94.3:33695)
0811 00:17:34.958736 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:34.958743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:34.958793 (+ 50us) server_negotiation.cc:407] Connection header received
0811 00:17:34.959046 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:34.959050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:34.959147 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:34.959308 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:17:34.959730 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.960426 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.961923 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:34.962322 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:34.962492 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:34.963101 (+ 609us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:17:34.963616 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:17:34.963841 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:34.963958 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 00:17:34.964006 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 00:17:34.972205 7470 raft_consensus.cc:1261] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804 [term 1 FOLLOWER]: Refusing update from remote peer 37595f65bb5f44798f58cbff8f57a342: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:17:34.972278 7731 raft_consensus.cc:1261] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c [term 1 FOLLOWER]: Refusing update from remote peer 37595f65bb5f44798f58cbff8f57a342: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:17:34.973250 8093 consensus_queue.cc:1035] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41baca490ebf449b8fd8690d3396ba1c" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 40859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:17:34.973613 8090 consensus_queue.cc:1035] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dbe85c72a8b450bb1c4eb0a8e200804" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 00:17:34.979015 7809 tablet.cc:2369] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:17:34.989380 8100 mvcc.cc:205] Tried to move back new op lower bound from 7317124935556829184 to 7317124935406616576. Current Snapshot: MvccSnapshot[applied={T|T < 7317124935556829184}]
I20260811 00:17:34.989632 8102 mvcc.cc:205] Tried to move back new op lower bound from 7317124935556829184 to 7317124935406616576. Current Snapshot: MvccSnapshot[applied={T|T < 7317124935556829184}]
I20260811 00:17:34.994264 8103 mvcc.cc:205] Tried to move back new op lower bound from 7317124935556829184 to 7317124935406616576. Current Snapshot: MvccSnapshot[applied={T|T < 7317124935556829184}]
I20260811 00:17:35.001578 7206 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:17:35.001899 7206 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:17:35.005661 7206 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b1eb21f3761e44729a5d9338e176479f: Sending DeleteTablet for 3 replicas of tablet 816031580b114b1b94319a5e1a18cc70
I20260811 00:17:35.006879 7919 tablet_service.cc:1514] Processing DeleteTablet for tablet 816031580b114b1b94319a5e1a18cc70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:17:34 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:37150
I20260811 00:17:35.006964 7431 tablet_service.cc:1514] Processing DeleteTablet for tablet 816031580b114b1b94319a5e1a18cc70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:17:34 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:44824
I20260811 00:17:35.007670 7698 tablet_service.cc:1514] Processing DeleteTablet for tablet 816031580b114b1b94319a5e1a18cc70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:17:34 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:49180
I20260811 00:17:35.007866 8118 tablet_replica.cc:330] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342: stopping tablet replica
I20260811 00:17:35.008355 8118 raft_consensus.cc:2229] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:17:35.009310 8118 raft_consensus.cc:2258] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:17:35.009732 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 7310
I20260811 00:17:35.010121 8120 tablet_replica.cc:330] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c: stopping tablet replica
I20260811 00:17:35.010309 8119 tablet_replica.cc:330] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804: stopping tablet replica
I20260811 00:17:35.010532 8120 raft_consensus.cc:2229] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:17:35.010672 8119 raft_consensus.cc:2229] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:17:35.011063 8120 raft_consensus.cc:2258] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:17:35.011109 8119 raft_consensus.cc:2258] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:17:35.014405 8119 ts_tablet_manager.cc:1905] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:17:35.014411 8118 ts_tablet_manager.cc:1905] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:17:35.020886 8120 ts_tablet_manager.cc:1905] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:17:35.021526 8119 ts_tablet_manager.cc:1918] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:17:35.021752 8119 log.cc:1196] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/816031580b114b1b94319a5e1a18cc70
I20260811 00:17:35.022236 8118 ts_tablet_manager.cc:1918] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:17:35.022415 8118 log.cc:1196] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/816031580b114b1b94319a5e1a18cc70
I20260811 00:17:35.022617 8119 ts_tablet_manager.cc:1939] T 816031580b114b1b94319a5e1a18cc70 P 4dbe85c72a8b450bb1c4eb0a8e200804: Deleting consensus metadata
I20260811 00:17:35.022969 8118 ts_tablet_manager.cc:1939] T 816031580b114b1b94319a5e1a18cc70 P 37595f65bb5f44798f58cbff8f57a342: Deleting consensus metadata
I20260811 00:17:35.024660 7189 catalog_manager.cc:4841] TS 37595f65bb5f44798f58cbff8f57a342 (127.7.94.3:33695): tablet 816031580b114b1b94319a5e1a18cc70 (table test-table [id=7e41ba9e8892445baa2c84c840d08744]) successfully deleted
I20260811 00:17:35.024691 7191 catalog_manager.cc:4841] TS 4dbe85c72a8b450bb1c4eb0a8e200804 (127.7.94.1:38655): tablet 816031580b114b1b94319a5e1a18cc70 (table test-table [id=7e41ba9e8892445baa2c84c840d08744]) successfully deleted
I20260811 00:17:35.029983 8120 ts_tablet_manager.cc:1918] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:17:35.030166 8120 log.cc:1196] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/816031580b114b1b94319a5e1a18cc70
I20260811 00:17:35.030754 8120 ts_tablet_manager.cc:1939] T 816031580b114b1b94319a5e1a18cc70 P 41baca490ebf449b8fd8690d3396ba1c: Deleting consensus metadata
I20260811 00:17:35.032605 7186 catalog_manager.cc:4841] TS 41baca490ebf449b8fd8690d3396ba1c (127.7.94.2:40859): tablet 816031580b114b1b94319a5e1a18cc70 (table test-table [id=7e41ba9e8892445baa2c84c840d08744]) successfully deleted
Aug 11 00:17:35 dist-test-slave-8hhm krb5kdc[7132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:17:35.042207 8073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:35.011320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44838 (local address 127.7.94.1:38655)
0811 00:17:35.011556 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:35.011563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:35.024993 (+ 13430us) server_negotiation.cc:407] Connection header received
0811 00:17:35.025321 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:35.025326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:35.025408 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:35.025482 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:35.027274 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:35.028420 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:35.030391 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:35.031061 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:35.031216 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:35.036261 (+ 5045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:35.036275 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:35.036291 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:35.036325 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:35.039328 (+ 3003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:35.040056 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:35.040063 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:35.040066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:35.040203 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:35.040601 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:35.040607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:35.040609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:35.040954 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:35.041315 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:35.041598 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 00:17:35.041811 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 00:17:35.042969 7542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:35.132651 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 7310
I20260811 00:17:35.166533 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 7571
Aug 11 00:17:35 dist-test-slave-8hhm krb5kdc[7132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.2@KRBTEST.COM
I20260811 00:17:35.183666 8066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:35.167366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49184 (local address 127.7.94.2:40859)
0811 00:17:35.167586 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:35.167592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:35.167616 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:17:35.167743 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:35.167749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:35.167836 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:35.167943 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:35.169435 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:35.170601 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:35.171868 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:35.172502 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:35.172644 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:35.176877 (+ 4233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:35.176898 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:35.176920 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:35.176968 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:35.180680 (+ 3712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:35.181400 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:35.181410 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:35.181414 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:35.181585 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:35.181988 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:35.181993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:35.181996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:35.182523 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:35.182839 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:35.183044 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 00:17:35.183275 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 00:17:35.184423 7799 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:35.273757 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 7571
I20260811 00:17:35.304543 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 7816
Aug 11 00:17:35 dist-test-slave-8hhm krb5kdc[7132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.3@KRBTEST.COM
I20260811 00:17:35.320986 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:35.305553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37166 (local address 127.7.94.3:33695)
0811 00:17:35.305762 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:35.305769 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:35.305793 (+ 24us) server_negotiation.cc:407] Connection header received
0811 00:17:35.305870 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:35.305875 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:35.305966 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:35.306087 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:35.307488 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:35.308359 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:35.309750 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:35.310236 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:35.310378 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:35.314905 (+ 4527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:35.314918 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:35.314934 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:35.314967 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:35.318118 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:35.318821 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:35.318828 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:35.318831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:35.318982 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:35.319406 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:35.319410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:35.319412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:35.319804 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:35.320140 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:35.320410 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 00:17:35.320628 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 00:17:35.321981 8032 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:35.443490 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 7816
I20260811 00:17:35.471578 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 7148
I20260811 00:17:35.472733 7251 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:35.581810 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 7148
2026-08-11T00:17:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4647 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T00:17:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:17:35Z Disabled control of system clock
I20260811 00:17:35.679150 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:41019
--webserver_interface=127.7.94.62
--webserver_port=0
--builtin_ntp_servers=127.7.94.20:44057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:41019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 00:17:36.014609 8134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:36.014881 8134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:36.014962 8134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:36.024869 8134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:17:36.024967 8134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:36.025056 8134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:36.025132 8134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:17:36.025195 8134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:17:36.041455 8134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:44057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:41019
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:41019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:36.045410 8134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:36.047808 8134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:36.064694 8146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:36.064843 8149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:36.064885 8147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:36.065790 8134 server_base.cc:1034] running on GCE node
I20260811 00:17:36.066642 8134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:36.068166 8134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:36.069406 8134 hybrid_clock.cc:648] HybridClock initialized: now 1786407456069346 us; error 40 us; skew 500 ppm
I20260811 00:17:36.069898 8134 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:17:36.073040 8134 webserver.cc:460] Webserver started at http://127.7.94.62:35043/ using document root <none> and password file <none>
I20260811 00:17:36.074054 8134 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:36.074209 8134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:36.074628 8134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:36.079279 8134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "ca4656584bbe430dbd7db24ce65f25d2"
format_stamp: "Formatted at 2026-08-11 00:17:36 on dist-test-slave-8hhm"
server_key: "2c303e0351bd494433c012b0967ad1e0"
server_key_iv: "2b82cd842c2127354870d038d76cfb85"
server_key_version: "encryptionkey@0"
I20260811 00:17:36.080507 8134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "ca4656584bbe430dbd7db24ce65f25d2"
format_stamp: "Formatted at 2026-08-11 00:17:36 on dist-test-slave-8hhm"
server_key: "2c303e0351bd494433c012b0967ad1e0"
server_key_iv: "2b82cd842c2127354870d038d76cfb85"
server_key_version: "encryptionkey@0"
I20260811 00:17:36.086957 8134 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260811 00:17:36.090315 8158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:36.091607 8134 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:17:36.091802 8134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "ca4656584bbe430dbd7db24ce65f25d2"
format_stamp: "Formatted at 2026-08-11 00:17:36 on dist-test-slave-8hhm"
server_key: "2c303e0351bd494433c012b0967ad1e0"
server_key_iv: "2b82cd842c2127354870d038d76cfb85"
server_key_version: "encryptionkey@0"
I20260811 00:17:36.092007 8134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:36.108681 8134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:36.109544 8134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:36.109834 8134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:36.127071 8134 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:41019
I20260811 00:17:36.127121 8245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:41019 every 8 connection(s)
I20260811 00:17:36.128382 8134 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:17:36.131932 8247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:36.138199 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 8134
I20260811 00:17:36.138360 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:17:36.138829 7544 external_mini_cluster.cc:1428] Setting key 061a14297b97636e19ea389abc50fbca
I20260811 00:17:36.142171 8247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2: Bootstrap starting.
I20260811 00:17:36.145457 8247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:36.146500 8247 log.cc:824] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:36.149381 8247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2: No bootstrap required, opened a new log
I20260811 00:17:36.152241 8251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:36.140662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54932 (local address 127.7.94.62:41019)
0811 00:17:36.141682 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:36.141692 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:36.141732 (+ 40us) server_negotiation.cc:407] Connection header received
0811 00:17:36.143468 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:36.143490 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:36.143925 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:36.144244 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:36.144861 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:36.146224 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:36.147949 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:36.148589 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:36.148710 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:36.150018 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:36.150025 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:36.150039 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:36.150570 (+ 531us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:36.150643 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:36.150667 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:36.150749 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:36.150984 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 00:17:36.151317 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":342,"threads_started":1}
I20260811 00:17:36.157878 8247 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4656584bbe430dbd7db24ce65f25d2" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41019 } }
I20260811 00:17:36.158233 8247 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:36.158349 8247 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca4656584bbe430dbd7db24ce65f25d2, State: Initialized, Role: FOLLOWER
I20260811 00:17:36.159003 8247 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4656584bbe430dbd7db24ce65f25d2" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41019 } }
I20260811 00:17:36.159327 8247 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:17:36.159467 8247 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:17:36.159621 8247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:36.161847 8247 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4656584bbe430dbd7db24ce65f25d2" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41019 } }
I20260811 00:17:36.162302 8247 leader_election.cc:302] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca4656584bbe430dbd7db24ce65f25d2; no voters:
I20260811 00:17:36.162843 8247 leader_election.cc:288] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:17:36.162987 8253 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:17:36.163722 8253 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2 [term 1 LEADER]: Becoming Leader. State: Replica: ca4656584bbe430dbd7db24ce65f25d2, State: Running, Role: LEADER
I20260811 00:17:36.164172 8253 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4656584bbe430dbd7db24ce65f25d2" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41019 } }
I20260811 00:17:36.164810 8247 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:17:36.167076 8259 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca4656584bbe430dbd7db24ce65f25d2. Latest consensus state: current_term: 1 leader_uuid: "ca4656584bbe430dbd7db24ce65f25d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4656584bbe430dbd7db24ce65f25d2" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41019 } } }
I20260811 00:17:36.167868 8259 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2 [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:36.167860 8254 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca4656584bbe430dbd7db24ce65f25d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca4656584bbe430dbd7db24ce65f25d2" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41019 } } }
I20260811 00:17:36.168555 8254 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2 [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:36.169608 8272 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:17:36.174995 8272 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:17:36.183768 8272 catalog_manager.cc:1347] Generated new cluster ID: 606329b0d7234fd7b3c7591a87111eea
I20260811 00:17:36.183890 8272 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:17:36.197319 8272 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:17:36.199159 8272 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:17:36.212493 8272 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2: Generated new TSK 0
I20260811 00:17:36.213513 8272 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:17:36.224627 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41019
--builtin_ntp_servers=127.7.94.20:44057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 00:17:36.567209 8289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:36.567534 8289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:36.567708 8289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:36.575107 8289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:36.575245 8289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:36.575430 8289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:17:36.584362 8289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:44057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41019
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:36.585631 8289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:36.587447 8289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:36.601029 8296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:36.601112 8295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:36.601691 8298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:36.601951 8289 server_base.cc:1034] running on GCE node
I20260811 00:17:36.602633 8289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:36.603583 8289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:36.604816 8289 hybrid_clock.cc:648] HybridClock initialized: now 1786407456604767 us; error 58 us; skew 500 ppm
I20260811 00:17:36.605175 8289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:17:36.607542 8289 webserver.cc:460] Webserver started at http://127.7.94.1:42451/ using document root <none> and password file <none>
I20260811 00:17:36.608273 8289 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:36.608408 8289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:36.608736 8289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:36.611687 8289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "37fc7656321d456ab7306ad350bfbadc"
format_stamp: "Formatted at 2026-08-11 00:17:36 on dist-test-slave-8hhm"
server_key: "ab4345ceb264c541df13101a9a6faf55"
server_key_iv: "e957bfc1b319027558d6aeb9eb527211"
server_key_version: "encryptionkey@0"
I20260811 00:17:36.612511 8289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "37fc7656321d456ab7306ad350bfbadc"
format_stamp: "Formatted at 2026-08-11 00:17:36 on dist-test-slave-8hhm"
server_key: "ab4345ceb264c541df13101a9a6faf55"
server_key_iv: "e957bfc1b319027558d6aeb9eb527211"
server_key_version: "encryptionkey@0"
I20260811 00:17:36.617808 8289 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 00:17:36.621330 8314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:36.622606 8289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:17:36.622789 8289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "37fc7656321d456ab7306ad350bfbadc"
format_stamp: "Formatted at 2026-08-11 00:17:36 on dist-test-slave-8hhm"
server_key: "ab4345ceb264c541df13101a9a6faf55"
server_key_iv: "e957bfc1b319027558d6aeb9eb527211"
server_key_version: "encryptionkey@0"
I20260811 00:17:36.623015 8289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:36.641721 8289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:36.642561 8289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:36.642848 8289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:36.643811 8289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:36.645313 8289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:17:36.645434 8289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:36.645565 8289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:36.645658 8289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:36.690385 8289 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:38953
I20260811 00:17:36.690433 8525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:38953 every 8 connection(s)
I20260811 00:17:36.691857 8289 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:17:36.696305 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 8289
I20260811 00:17:36.696558 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:17:36.697078 7544 external_mini_cluster.cc:1428] Setting key 81696fe4984eef6bf5393a30b045857f
I20260811 00:17:36.700017 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41019
--builtin_ntp_servers=127.7.94.20:44057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 00:17:36.704622 8530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:36.694477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:40635 (local address 127.7.94.62:41019)
0811 00:17:36.694921 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:36.694926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:36.696070 (+ 1144us) server_negotiation.cc:407] Connection header received
0811 00:17:36.698159 (+ 2089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:36.698164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:36.698251 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:36.698368 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:36.699522 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:36.700302 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:36.701866 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:36.702269 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:36.702401 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:36.703793 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:36.703797 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:36.703800 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:36.703817 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:36.703858 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:36.703869 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:36.703964 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:36.704190 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 00:17:36.704287 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":241,"threads_started":1}
I20260811 00:17:36.705614 8533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:36.694928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41019 (local address 127.7.94.1:40635)
0811 00:17:36.695907 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:36.695958 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:36.697875 (+ 1917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:36.698605 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:36.698613 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:36.698701 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:17:36.699360 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:36.699378 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:36.700451 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:36.700455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:36.701729 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:36.701741 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:36.702497 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:36.702505 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:36.702752 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:36.703478 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:17:36.703481 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:17:36.703517 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:17:36.703592 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:17:36.703965 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:17:36.703979 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:17:36.703988 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 00:17:36.704170 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 00:17:36.704540 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":315,"threads_started":1}
I20260811 00:17:36.707185 8526 heartbeater.cc:347] Connected to a master server at 127.7.94.62:41019
I20260811 00:17:36.707461 8526 heartbeater.cc:464] Registering TS with master...
I20260811 00:17:36.708154 8526 heartbeater.cc:511] Master 127.7.94.62:41019 requested a full tablet report, sending...
I20260811 00:17:36.710129 8182 ts_manager.cc:194] Registered new tserver with Master: 37fc7656321d456ab7306ad350bfbadc (127.7.94.1:38953)
I20260811 00:17:36.711731 8182 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.94.1:40635
W20260811 00:17:37.057628 8534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:37.057953 8534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:37.058138 8534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:37.065701 8534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:37.065826 8534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:37.066025 8534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:17:37.075304 8534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:44057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41019
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:37.076673 8534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:37.078591 8534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:37.092561 8549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:37.092643 8544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:37.092612 8545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:37.093405 8534 server_base.cc:1034] running on GCE node
I20260811 00:17:37.094041 8534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:37.095014 8534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:37.096213 8534 hybrid_clock.cc:648] HybridClock initialized: now 1786407457096160 us; error 56 us; skew 500 ppm
I20260811 00:17:37.096529 8534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:17:37.098788 8534 webserver.cc:460] Webserver started at http://127.7.94.2:34163/ using document root <none> and password file <none>
I20260811 00:17:37.099453 8534 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:37.099562 8534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:37.099901 8534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:37.102681 8534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "0bb032124963426d90de7c8fa4a89bbd"
format_stamp: "Formatted at 2026-08-11 00:17:37 on dist-test-slave-8hhm"
server_key: "8314baf5ea39bbf705b9f327dce0ebae"
server_key_iv: "cdebf39cfd56acdd30aae9b91b94cea1"
server_key_version: "encryptionkey@0"
I20260811 00:17:37.103468 8534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "0bb032124963426d90de7c8fa4a89bbd"
format_stamp: "Formatted at 2026-08-11 00:17:37 on dist-test-slave-8hhm"
server_key: "8314baf5ea39bbf705b9f327dce0ebae"
server_key_iv: "cdebf39cfd56acdd30aae9b91b94cea1"
server_key_version: "encryptionkey@0"
I20260811 00:17:37.108237 8534 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260811 00:17:37.111729 8565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:37.113039 8534 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:17:37.113266 8534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "0bb032124963426d90de7c8fa4a89bbd"
format_stamp: "Formatted at 2026-08-11 00:17:37 on dist-test-slave-8hhm"
server_key: "8314baf5ea39bbf705b9f327dce0ebae"
server_key_iv: "cdebf39cfd56acdd30aae9b91b94cea1"
server_key_version: "encryptionkey@0"
I20260811 00:17:37.113514 8534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:37.132419 8534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:37.133325 8534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:37.133598 8534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:37.134665 8534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:37.136426 8534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:17:37.136531 8534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:37.136622 8534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:37.136689 8534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:37.170640 8534 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:36287
I20260811 00:17:37.170670 8747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:36287 every 8 connection(s)
I20260811 00:17:37.172004 8534 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:17:37.173372 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 8534
I20260811 00:17:37.173564 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:17:37.174170 7544 external_mini_cluster.cc:1428] Setting key a93e90dfc01391dd2f93d90df6cac184
I20260811 00:17:37.177526 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41019
--builtin_ntp_servers=127.7.94.20:44057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 00:17:37.187429 8530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:37.174916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:44825 (local address 127.7.94.62:41019)
0811 00:17:37.175121 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:37.175125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:37.176751 (+ 1626us) server_negotiation.cc:407] Connection header received
0811 00:17:37.179532 (+ 2781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:37.179536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:37.179600 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:37.179680 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:37.181056 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.181779 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.184211 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.184800 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.184919 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:37.186562 (+ 1643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:37.186569 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:37.186575 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:37.186596 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:37.186634 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:37.186649 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:37.186726 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:37.186962 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 00:17:37.187061 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 00:17:37.188727 8752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:37.175538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41019 (local address 127.7.94.2:44825)
0811 00:17:37.176579 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:37.176628 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:37.179235 (+ 2607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:37.179948 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:37.179960 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:37.180069 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:17:37.180863 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:37.180891 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.181929 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.181936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:37.184055 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:37.184072 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.185025 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.185033 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:37.185388 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:37.186198 (+ 810us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:17:37.186204 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:17:37.186271 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:17:37.186380 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:17:37.186747 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:17:37.186759 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:17:37.186769 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 00:17:37.186941 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 00:17:37.187346 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":308,"threads_started":1}
I20260811 00:17:37.190090 8749 heartbeater.cc:347] Connected to a master server at 127.7.94.62:41019
I20260811 00:17:37.190409 8749 heartbeater.cc:464] Registering TS with master...
I20260811 00:17:37.191358 8749 heartbeater.cc:511] Master 127.7.94.62:41019 requested a full tablet report, sending...
I20260811 00:17:37.192802 8182 ts_manager.cc:194] Registered new tserver with Master: 0bb032124963426d90de7c8fa4a89bbd (127.7.94.2:36287)
I20260811 00:17:37.193662 8182 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.94.2:44825
W20260811 00:17:37.518857 8753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:37.519111 8753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:37.519275 8753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:37.526270 8753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:37.526381 8753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:37.526533 8753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:17:37.535691 8753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:44057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:41019
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:37.536933 8753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:37.538750 8753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:37.552100 8774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:37.552055 8775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:37.552018 8778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:37.552773 8753 server_base.cc:1034] running on GCE node
I20260811 00:17:37.553442 8753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:37.554323 8753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:37.555555 8753 hybrid_clock.cc:648] HybridClock initialized: now 1786407457555515 us; error 45 us; skew 500 ppm
I20260811 00:17:37.555872 8753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:17:37.558068 8753 webserver.cc:460] Webserver started at http://127.7.94.3:37811/ using document root <none> and password file <none>
I20260811 00:17:37.558739 8753 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:37.558872 8753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:37.559199 8753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:37.562260 8753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "518f9c34d4d641098504d47c8e80a98c"
format_stamp: "Formatted at 2026-08-11 00:17:37 on dist-test-slave-8hhm"
server_key: "b15e665dc3051cddb5320f815d6e0a1d"
server_key_iv: "5012268e479da07b15f6da79c0644112"
server_key_version: "encryptionkey@0"
I20260811 00:17:37.563037 8753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "518f9c34d4d641098504d47c8e80a98c"
format_stamp: "Formatted at 2026-08-11 00:17:37 on dist-test-slave-8hhm"
server_key: "b15e665dc3051cddb5320f815d6e0a1d"
server_key_iv: "5012268e479da07b15f6da79c0644112"
server_key_version: "encryptionkey@0"
I20260811 00:17:37.568249 8753 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:17:37.572094 8789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:37.573385 8753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:17:37.573556 8753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "518f9c34d4d641098504d47c8e80a98c"
format_stamp: "Formatted at 2026-08-11 00:17:37 on dist-test-slave-8hhm"
server_key: "b15e665dc3051cddb5320f815d6e0a1d"
server_key_iv: "5012268e479da07b15f6da79c0644112"
server_key_version: "encryptionkey@0"
I20260811 00:17:37.573776 8753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:37.610318 8753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:37.611213 8753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:37.611500 8753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:37.612495 8753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:37.613999 8753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:17:37.614135 8753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:37.614260 8753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:37.614351 8753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:37.647137 8753 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:44709
I20260811 00:17:37.647188 8982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:44709 every 8 connection(s)
I20260811 00:17:37.648440 8753 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:17:37.652701 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 8753
I20260811 00:17:37.652890 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:17:37.653414 7544 external_mini_cluster.cc:1428] Setting key 9b744c77e92f36f79f1825ab77442037
I20260811 00:17:37.661058 8530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:37.651017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:38219 (local address 127.7.94.62:41019)
0811 00:17:37.651227 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:37.651233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:37.652644 (+ 1411us) server_negotiation.cc:407] Connection header received
0811 00:17:37.654847 (+ 2203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:37.654852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:37.654944 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:37.655056 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:37.656109 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.656841 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.658394 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.658824 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.658969 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:37.660201 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:37.660206 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:37.660209 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:37.660245 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:37.660277 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:37.660289 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:37.660388 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:37.660625 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 00:17:37.660722 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 00:17:37.661998 8986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:37.651565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41019 (local address 127.7.94.3:38219)
0811 00:17:37.652464 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:37.652507 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:37.654567 (+ 2060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:37.655309 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:37.655317 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:37.655406 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:17:37.655939 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:37.655955 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.657002 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.657008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:37.658248 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:37.658261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.659072 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.659079 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:37.659301 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:37.659891 (+ 590us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:17:37.659894 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:17:37.659929 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:17:37.660001 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:17:37.660390 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:17:37.660404 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:17:37.660415 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 00:17:37.660602 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 00:17:37.660967 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":299,"threads_started":1}
I20260811 00:17:37.663252 8983 heartbeater.cc:347] Connected to a master server at 127.7.94.62:41019
I20260811 00:17:37.663501 8983 heartbeater.cc:464] Registering TS with master...
I20260811 00:17:37.664142 8983 heartbeater.cc:511] Master 127.7.94.62:41019 requested a full tablet report, sending...
I20260811 00:17:37.665477 8182 ts_manager.cc:194] Registered new tserver with Master: 518f9c34d4d641098504d47c8e80a98c (127.7.94.3:44709)
I20260811 00:17:37.666363 8182 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.94.3:38219
I20260811 00:17:37.668260 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:17:37.678095 8530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:37.672302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54944 (local address 127.7.94.62:41019)
0811 00:17:37.672425 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:37.672429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:37.672882 (+ 453us) server_negotiation.cc:407] Connection header received
0811 00:17:37.673008 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:37.673011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:37.673079 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:37.673157 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:37.673632 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.674293 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.675653 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.676035 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.676124 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:37.677356 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:37.677360 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:37.677363 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:37.677379 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:37.677415 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:37.677427 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:37.677534 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:37.677705 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 00:17:37.677773 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 00:17:37.681857 8182 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:17:37.688658 8182 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:17:37.714663 8526 heartbeater.cc:503] Master 127.7.94.62:41019 was elected leader, sending a full tablet report...
I20260811 00:17:37.722050 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:37.700768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55676 (local address 127.7.94.3:44709)
0811 00:17:37.712123 (+ 11355us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:37.712128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:37.712156 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:17:37.712237 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:37.712242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:37.712421 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:37.712625 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:37.713435 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.715916 (+ 2481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.719117 (+ 3201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.720421 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.720567 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:37.720840 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:37.721497 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 00:17:37.721672 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11188,"thread_start_us":11039,"threads_started":1}
I20260811 00:17:37.723124 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:37.700568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:44709 (local address 127.0.0.1:55676)
0811 00:17:37.707541 (+ 6973us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:37.707594 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:37.707801 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:37.712638 (+ 4837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:37.712646 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:37.712679 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:37.713145 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:37.713164 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.716067 (+ 2903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.716073 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:37.718891 (+ 2818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:37.718907 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.720627 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.720633 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:37.721192 (+ 559us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:37.721296 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 00:17:37.721435 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 00:17:37.722714 (+ 1279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6643,"mutex_wait_us":1067,"thread_start_us":6458,"threads_started":1}
I20260811 00:17:37.726506 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:37.713041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38953 (local address 127.0.0.1:48620)
0811 00:17:37.718299 (+ 5258us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:37.718329 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:37.718437 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:37.719057 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:37.719063 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:37.719090 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:37.719572 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:37.719581 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.721028 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.721033 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:37.723312 (+ 2279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:37.723326 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.725734 (+ 2408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.725740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:37.725913 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:37.725951 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 00:17:37.726018 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 00:17:37.726146 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5124,"mutex_wait_us":29,"thread_start_us":4358,"threads_started":1}
I20260811 00:17:37.726696 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:37.701235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57836 (local address 127.7.94.2:36287)
0811 00:17:37.711735 (+ 10500us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:37.711749 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:37.717990 (+ 6241us) server_negotiation.cc:407] Connection header received
0811 00:17:37.718264 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:37.718272 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:37.718522 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:37.718753 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:37.719711 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.721275 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.723721 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.725078 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.725196 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:37.725461 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:37.726128 (+ 667us) server_negotiation.cc:299] Negotiation successful
0811 00:17:37.726345 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10301,"thread_start_us":10170,"threads_started":1}
I20260811 00:17:37.727177 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:37.700656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:36287 (local address 127.0.0.1:57836)
0811 00:17:37.717848 (+ 17192us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:37.717866 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:37.717999 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:37.718937 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:37.718940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:37.718963 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:37.719500 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:37.719516 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.721399 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.721405 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:37.723514 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:37.723531 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.725620 (+ 2089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.725629 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:37.725828 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:37.725861 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 00:17:37.725948 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 00:17:37.726847 (+ 899us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17066,"mutex_wait_us":863,"spinlock_wait_cycles":1280,"thread_start_us":16907,"threads_started":1}
I20260811 00:17:37.727596 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:37.708399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48620 (local address 127.7.94.1:38953)
0811 00:17:37.717744 (+ 9345us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:37.717752 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:37.718563 (+ 811us) server_negotiation.cc:407] Connection header received
0811 00:17:37.718659 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:37.718665 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:37.718860 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:37.719063 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:37.719748 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.720867 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.723561 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.725031 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.725174 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:37.725496 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:37.727037 (+ 1541us) server_negotiation.cc:299] Negotiation successful
0811 00:17:37.727214 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8762,"thread_start_us":8616,"threads_started":1}
I20260811 00:17:37.727243 8862 tablet_service.cc:1467] Processing CreateTablet for tablet 6cac6bf934384d3d958f5e138cad84cf (DEFAULT_TABLE table=test-table [id=fd7bfab7abc74d06b5285e6d2f3dd58f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:17:37.728889 8862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cac6bf934384d3d958f5e138cad84cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:37.731063 8638 tablet_service.cc:1467] Processing CreateTablet for tablet 6cac6bf934384d3d958f5e138cad84cf (DEFAULT_TABLE table=test-table [id=fd7bfab7abc74d06b5285e6d2f3dd58f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:17:37.731063 8392 tablet_service.cc:1467] Processing CreateTablet for tablet 6cac6bf934384d3d958f5e138cad84cf (DEFAULT_TABLE table=test-table [id=fd7bfab7abc74d06b5285e6d2f3dd58f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:17:37.732532 8638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cac6bf934384d3d958f5e138cad84cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:37.732545 8392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cac6bf934384d3d958f5e138cad84cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:37.742017 9011 tablet_bootstrap.cc:492] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c: Bootstrap starting.
I20260811 00:17:37.745323 9011 tablet_bootstrap.cc:654] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:37.746457 9011 log.cc:824] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:37.747231 9013 tablet_bootstrap.cc:492] T 6cac6bf934384d3d958f5e138cad84cf P 37fc7656321d456ab7306ad350bfbadc: Bootstrap starting.
I20260811 00:17:37.750198 9016 tablet_bootstrap.cc:492] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd: Bootstrap starting.
I20260811 00:17:37.751225 9013 tablet_bootstrap.cc:654] T 6cac6bf934384d3d958f5e138cad84cf P 37fc7656321d456ab7306ad350bfbadc: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:37.752533 9013 log.cc:824] T 6cac6bf934384d3d958f5e138cad84cf P 37fc7656321d456ab7306ad350bfbadc: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:37.753799 9016 tablet_bootstrap.cc:654] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:37.754523 9011 tablet_bootstrap.cc:492] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c: No bootstrap required, opened a new log
I20260811 00:17:37.754786 9011 ts_tablet_manager.cc:1397] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.002s
I20260811 00:17:37.754983 9016 log.cc:824] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:37.755651 9013 tablet_bootstrap.cc:492] T 6cac6bf934384d3d958f5e138cad84cf P 37fc7656321d456ab7306ad350bfbadc: No bootstrap required, opened a new log
I20260811 00:17:37.755918 9013 ts_tablet_manager.cc:1397] T 6cac6bf934384d3d958f5e138cad84cf P 37fc7656321d456ab7306ad350bfbadc: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260811 00:17:37.758544 9016 tablet_bootstrap.cc:492] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd: No bootstrap required, opened a new log
I20260811 00:17:37.758788 9016 ts_tablet_manager.cc:1397] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 00:17:37.762641 9011 raft_consensus.cc:348] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518f9c34d4d641098504d47c8e80a98c" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44709 } } peers { permanent_uuid: "0bb032124963426d90de7c8fa4a89bbd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36287 } } peers { permanent_uuid: "37fc7656321d456ab7306ad350bfbadc" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38953 } }
I20260811 00:17:37.762998 9011 raft_consensus.cc:374] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:37.763103 9011 raft_consensus.cc:729] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 518f9c34d4d641098504d47c8e80a98c, State: Initialized, Role: FOLLOWER
I20260811 00:17:37.763721 9011 consensus_queue.cc:260] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518f9c34d4d641098504d47c8e80a98c" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44709 } } peers { permanent_uuid: "0bb032124963426d90de7c8fa4a89bbd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36287 } } peers { permanent_uuid: "37fc7656321d456ab7306ad350bfbadc" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38953 } }
I20260811 00:17:37.764816 8983 heartbeater.cc:503] Master 127.7.94.62:41019 was elected leader, sending a full tablet report...
I20260811 00:17:37.765479 9011 ts_tablet_manager.cc:1428] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c: Time spent starting tablet: real 0.011s user 0.011s sys 0.001s
I20260811 00:17:37.767381 9013 raft_consensus.cc:348] T 6cac6bf934384d3d958f5e138cad84cf P 37fc7656321d456ab7306ad350bfbadc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518f9c34d4d641098504d47c8e80a98c" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44709 } } peers { permanent_uuid: "0bb032124963426d90de7c8fa4a89bbd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36287 } } peers { permanent_uuid: "37fc7656321d456ab7306ad350bfbadc" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38953 } }
I20260811 00:17:37.767843 9013 raft_consensus.cc:374] T 6cac6bf934384d3d958f5e138cad84cf P 37fc7656321d456ab7306ad350bfbadc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:37.767975 9013 raft_consensus.cc:729] T 6cac6bf934384d3d958f5e138cad84cf P 37fc7656321d456ab7306ad350bfbadc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37fc7656321d456ab7306ad350bfbadc, State: Initialized, Role: FOLLOWER
I20260811 00:17:37.768568 9013 consensus_queue.cc:260] T 6cac6bf934384d3d958f5e138cad84cf P 37fc7656321d456ab7306ad350bfbadc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518f9c34d4d641098504d47c8e80a98c" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44709 } } peers { permanent_uuid: "0bb032124963426d90de7c8fa4a89bbd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36287 } } peers { permanent_uuid: "37fc7656321d456ab7306ad350bfbadc" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38953 } }
I20260811 00:17:37.768966 9016 raft_consensus.cc:348] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518f9c34d4d641098504d47c8e80a98c" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44709 } } peers { permanent_uuid: "0bb032124963426d90de7c8fa4a89bbd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36287 } } peers { permanent_uuid: "37fc7656321d456ab7306ad350bfbadc" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38953 } }
I20260811 00:17:37.769452 9016 raft_consensus.cc:374] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:37.769598 9016 raft_consensus.cc:729] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bb032124963426d90de7c8fa4a89bbd, State: Initialized, Role: FOLLOWER
I20260811 00:17:37.770375 9016 consensus_queue.cc:260] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518f9c34d4d641098504d47c8e80a98c" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44709 } } peers { permanent_uuid: "0bb032124963426d90de7c8fa4a89bbd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36287 } } peers { permanent_uuid: "37fc7656321d456ab7306ad350bfbadc" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38953 } }
I20260811 00:17:37.771099 9013 ts_tablet_manager.cc:1428] T 6cac6bf934384d3d958f5e138cad84cf P 37fc7656321d456ab7306ad350bfbadc: Time spent starting tablet: real 0.015s user 0.006s sys 0.009s
I20260811 00:17:37.772472 9016 ts_tablet_manager.cc:1428] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260811 00:17:37.772866 8749 heartbeater.cc:503] Master 127.7.94.62:41019 was elected leader, sending a full tablet report...
W20260811 00:17:37.901885 8985 tablet.cc:2369] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:17:37.918825 9023 raft_consensus.cc:482] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:17:37.919168 9023 raft_consensus.cc:504] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518f9c34d4d641098504d47c8e80a98c" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44709 } } peers { permanent_uuid: "0bb032124963426d90de7c8fa4a89bbd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36287 } } peers { permanent_uuid: "37fc7656321d456ab7306ad350bfbadc" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38953 } }
I20260811 00:17:37.920882 9023 leader_election.cc:288] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 518f9c34d4d641098504d47c8e80a98c (127.7.94.3:44709), 37fc7656321d456ab7306ad350bfbadc (127.7.94.1:38953)
W20260811 00:17:37.924209 8751 tablet.cc:2369] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:17:37.928516 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:37.921401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:48451 (local address 127.7.94.3:44709)
0811 00:17:37.921648 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:37.921658 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:37.922044 (+ 386us) server_negotiation.cc:407] Connection header received
0811 00:17:37.922339 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:37.922350 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:37.922460 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:37.922579 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:37.923272 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.924262 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.926493 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.927498 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.927595 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:37.927733 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:37.928097 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 00:17:37.928226 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:17:37.928546 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:37.921316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:44709 (local address 127.7.94.2:48451)
0811 00:17:37.921931 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:37.921947 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:37.922148 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:37.922748 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:37.922757 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:37.922787 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:37.923150 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:37.923160 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.924426 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.924431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:37.926303 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:37.926318 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.927731 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.927739 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:37.927933 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:37.927966 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 00:17:37.928055 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 00:17:37.928192 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":2,"thread_start_us":332,"threads_started":1}
I20260811 00:17:37.928877 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:37.922381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:57585 (local address 127.7.94.1:38953)
0811 00:17:37.922595 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:37.922603 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:37.923006 (+ 403us) server_negotiation.cc:407] Connection header received
0811 00:17:37.923224 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:37.923229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:37.923319 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:37.923422 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:37.923904 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.924946 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.926906 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.927957 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.928085 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:37.928177 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:37.928568 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 00:17:37.928718 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 00:17:37.929028 9026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:37.922104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38953 (local address 127.7.94.2:57585)
0811 00:17:37.922897 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:37.922910 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:37.923062 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:37.923441 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:37.923444 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:37.923464 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:37.923762 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:37.923769 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.925084 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.925087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:37.926725 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:37.926739 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.928164 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.928170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:37.928339 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:37.928372 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 00:17:37.928529 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 00:17:37.928690 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"mutex_wait_us":18,"thread_start_us":511,"threads_started":1}
I20260811 00:17:37.929975 8900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cac6bf934384d3d958f5e138cad84cf" candidate_uuid: "0bb032124963426d90de7c8fa4a89bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "518f9c34d4d641098504d47c8e80a98c" is_pre_election: true
I20260811 00:17:37.930357 8441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cac6bf934384d3d958f5e138cad84cf" candidate_uuid: "0bb032124963426d90de7c8fa4a89bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37fc7656321d456ab7306ad350bfbadc" is_pre_election: true
I20260811 00:17:37.930578 8900 raft_consensus.cc:2454] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bb032124963426d90de7c8fa4a89bbd in term 0.
I20260811 00:17:37.930994 8441 raft_consensus.cc:2454] T 6cac6bf934384d3d958f5e138cad84cf P 37fc7656321d456ab7306ad350bfbadc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bb032124963426d90de7c8fa4a89bbd in term 0.
I20260811 00:17:37.931345 8567 leader_election.cc:302] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bb032124963426d90de7c8fa4a89bbd, 518f9c34d4d641098504d47c8e80a98c; no voters:
I20260811 00:17:37.931703 9023 raft_consensus.cc:2781] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:17:37.931882 9023 raft_consensus.cc:482] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:17:37.932010 9023 raft_consensus.cc:3037] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:37.934331 9023 raft_consensus.cc:504] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518f9c34d4d641098504d47c8e80a98c" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44709 } } peers { permanent_uuid: "0bb032124963426d90de7c8fa4a89bbd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36287 } } peers { permanent_uuid: "37fc7656321d456ab7306ad350bfbadc" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38953 } }
I20260811 00:17:37.934859 9023 leader_election.cc:288] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [CANDIDATE]: Term 1 election: Requested vote from peers 518f9c34d4d641098504d47c8e80a98c (127.7.94.3:44709), 37fc7656321d456ab7306ad350bfbadc (127.7.94.1:38953)
I20260811 00:17:37.935364 8441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cac6bf934384d3d958f5e138cad84cf" candidate_uuid: "0bb032124963426d90de7c8fa4a89bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37fc7656321d456ab7306ad350bfbadc"
I20260811 00:17:37.935343 8900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cac6bf934384d3d958f5e138cad84cf" candidate_uuid: "0bb032124963426d90de7c8fa4a89bbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "518f9c34d4d641098504d47c8e80a98c"
I20260811 00:17:37.935554 8900 raft_consensus.cc:3037] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:37.935552 8441 raft_consensus.cc:3037] T 6cac6bf934384d3d958f5e138cad84cf P 37fc7656321d456ab7306ad350bfbadc [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:37.938674 8441 raft_consensus.cc:2454] T 6cac6bf934384d3d958f5e138cad84cf P 37fc7656321d456ab7306ad350bfbadc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bb032124963426d90de7c8fa4a89bbd in term 1.
I20260811 00:17:37.938673 8900 raft_consensus.cc:2454] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bb032124963426d90de7c8fa4a89bbd in term 1.
I20260811 00:17:37.939042 8567 leader_election.cc:302] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bb032124963426d90de7c8fa4a89bbd, 37fc7656321d456ab7306ad350bfbadc; no voters:
I20260811 00:17:37.939297 9023 raft_consensus.cc:2781] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:17:37.939723 9023 raft_consensus.cc:686] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [term 1 LEADER]: Becoming Leader. State: Replica: 0bb032124963426d90de7c8fa4a89bbd, State: Running, Role: LEADER
I20260811 00:17:37.940162 9023 consensus_queue.cc:237] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518f9c34d4d641098504d47c8e80a98c" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44709 } } peers { permanent_uuid: "0bb032124963426d90de7c8fa4a89bbd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36287 } } peers { permanent_uuid: "37fc7656321d456ab7306ad350bfbadc" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38953 } }
W20260811 00:17:37.944391 8527 tablet.cc:2369] T 6cac6bf934384d3d958f5e138cad84cf P 37fc7656321d456ab7306ad350bfbadc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:17:37.944461 8182 catalog_manager.cc:5495] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bb032124963426d90de7c8fa4a89bbd (127.7.94.2). New cstate: current_term: 1 leader_uuid: "0bb032124963426d90de7c8fa4a89bbd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "518f9c34d4d641098504d47c8e80a98c" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bb032124963426d90de7c8fa4a89bbd" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 36287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37fc7656321d456ab7306ad350bfbadc" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38953 } health_report { overall_health: UNKNOWN } } }
I20260811 00:17:37.964514 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:37.959577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57852 (local address 127.7.94.2:36287)
0811 00:17:37.959761 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:37.959765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:37.959780 (+ 15us) server_negotiation.cc:407] Connection header received
0811 00:17:37.959850 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:37.959854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:37.959936 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:37.960054 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:17:37.960397 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.961167 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.962597 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:37.963000 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:37.963091 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:37.963563 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:17:37.964023 (+ 460us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:17:37.964164 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:37.964288 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 00:17:37.964322 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 00:17:37.972625 8441 raft_consensus.cc:1261] T 6cac6bf934384d3d958f5e138cad84cf P 37fc7656321d456ab7306ad350bfbadc [term 1 FOLLOWER]: Refusing update from remote peer 0bb032124963426d90de7c8fa4a89bbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:17:37.972632 8900 raft_consensus.cc:1261] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c [term 1 FOLLOWER]: Refusing update from remote peer 0bb032124963426d90de7c8fa4a89bbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:17:37.973557 9028 consensus_queue.cc:1035] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "518f9c34d4d641098504d47c8e80a98c" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 44709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:17:37.973925 9023 consensus_queue.cc:1035] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "37fc7656321d456ab7306ad350bfbadc" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 38953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:17:37.985697 9033 mvcc.cc:205] Tried to move back new op lower bound from 7317124947845664768 to 7317124947725963264. Current Snapshot: MvccSnapshot[applied={T|T < 7317124947845664768}]
I20260811 00:17:37.986392 9035 mvcc.cc:205] Tried to move back new op lower bound from 7317124947845664768 to 7317124947725963264. Current Snapshot: MvccSnapshot[applied={T|T < 7317124947845664768}]
I20260811 00:17:38.000439 8182 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:17:38.000717 8182 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:17:38.004458 8182 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca4656584bbe430dbd7db24ce65f25d2: Sending DeleteTablet for 3 replicas of tablet 6cac6bf934384d3d958f5e138cad84cf
I20260811 00:17:38.005730 8862 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cac6bf934384d3d958f5e138cad84cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:17:37 UTC) from {username='slave'} at 127.0.0.1:55676
I20260811 00:17:38.005749 8638 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cac6bf934384d3d958f5e138cad84cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:17:37 UTC) from {username='slave'} at 127.0.0.1:57836
I20260811 00:17:38.006109 8392 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cac6bf934384d3d958f5e138cad84cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:17:37 UTC) from {username='slave'} at 127.0.0.1:48620
I20260811 00:17:38.007936 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 8289
I20260811 00:17:38.021968 9051 tablet_replica.cc:330] T 6cac6bf934384d3d958f5e138cad84cf P 37fc7656321d456ab7306ad350bfbadc: stopping tablet replica
I20260811 00:17:38.022765 9051 raft_consensus.cc:2229] T 6cac6bf934384d3d958f5e138cad84cf P 37fc7656321d456ab7306ad350bfbadc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:17:38.023540 9051 raft_consensus.cc:2258] T 6cac6bf934384d3d958f5e138cad84cf P 37fc7656321d456ab7306ad350bfbadc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:17:38.025574 9048 tablet_replica.cc:330] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd: stopping tablet replica
I20260811 00:17:38.026154 9048 raft_consensus.cc:2229] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:17:38.026991 9048 raft_consensus.cc:2258] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:17:38.028324 9051 ts_tablet_manager.cc:1905] T 6cac6bf934384d3d958f5e138cad84cf P 37fc7656321d456ab7306ad350bfbadc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:17:38.028995 9044 tablet_replica.cc:330] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c: stopping tablet replica
I20260811 00:17:38.029261 9048 ts_tablet_manager.cc:1905] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:17:38.029479 9044 raft_consensus.cc:2229] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:17:38.030059 9044 raft_consensus.cc:2258] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:17:38.030833 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:38.009604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48628 (local address 127.7.94.1:38953)
0811 00:17:38.009829 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:38.009838 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:38.014976 (+ 5138us) server_negotiation.cc:407] Connection header received
0811 00:17:38.015920 (+ 944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:38.015929 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:38.016063 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:38.016166 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:38.017614 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:38.019072 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:38.024511 (+ 5439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:38.025367 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:38.025474 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:38.029157 (+ 3683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:38.029168 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:38.029184 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:38.029921 (+ 737us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:38.029980 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:38.030011 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:38.030106 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:38.030343 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 00:17:38.030458 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 00:17:38.031985 8518 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:38.033895 9044 ts_tablet_manager.cc:1905] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:17:38.037560 9048 ts_tablet_manager.cc:1918] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:17:38.037840 9048 log.cc:1196] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/6cac6bf934384d3d958f5e138cad84cf
I20260811 00:17:38.038528 9048 ts_tablet_manager.cc:1939] T 6cac6bf934384d3d958f5e138cad84cf P 0bb032124963426d90de7c8fa4a89bbd: Deleting consensus metadata
I20260811 00:17:38.040401 8159 catalog_manager.cc:4841] TS 0bb032124963426d90de7c8fa4a89bbd (127.7.94.2:36287): tablet 6cac6bf934384d3d958f5e138cad84cf (table test-table [id=fd7bfab7abc74d06b5285e6d2f3dd58f]) successfully deleted
I20260811 00:17:38.040699 9044 ts_tablet_manager.cc:1918] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:17:38.041507 9044 log.cc:1196] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/6cac6bf934384d3d958f5e138cad84cf
I20260811 00:17:38.042153 9044 ts_tablet_manager.cc:1939] T 6cac6bf934384d3d958f5e138cad84cf P 518f9c34d4d641098504d47c8e80a98c: Deleting consensus metadata
I20260811 00:17:38.044062 8162 catalog_manager.cc:4841] TS 518f9c34d4d641098504d47c8e80a98c (127.7.94.3:44709): tablet 6cac6bf934384d3d958f5e138cad84cf (table test-table [id=fd7bfab7abc74d06b5285e6d2f3dd58f]) successfully deleted
I20260811 00:17:38.130514 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 8289
I20260811 00:17:38.161595 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 8534
W20260811 00:17:38.161754 8162 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.94.1:38953 (error 108)
I20260811 00:17:38.169890 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:38.169327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38953 (local address 127.0.0.1:48642)
0811 00:17:38.169538 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:38.169689 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:38953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 00:17:38.170373 8162 catalog_manager.cc:4584] TS 37fc7656321d456ab7306ad350bfbadc (127.7.94.1:38953): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6cac6bf934384d3d958f5e138cad84cf: Network error: Client connection negotiation failed: client connection to 127.7.94.1:38953: connect: Connection refused (error 111)
I20260811 00:17:38.172183 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:38.162962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57856 (local address 127.7.94.2:36287)
0811 00:17:38.163154 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:38.163160 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:38.163440 (+ 280us) server_negotiation.cc:407] Connection header received
0811 00:17:38.163677 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:38.163683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:38.163782 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:38.163854 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:38.164516 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:38.165716 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:38.167596 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:38.168275 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:38.168406 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:38.170526 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:38.170539 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:38.170557 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:38.171296 (+ 739us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:38.171343 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:38.171364 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:38.171423 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:38.171643 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 00:17:38.171751 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 00:17:38.172943 8736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:38.224491 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:38.224037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38953 (local address 127.0.0.1:48644)
0811 00:17:38.224240 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:38.224347 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:38953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 00:17:38.225571 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:38.225239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38953 (local address 127.0.0.1:48646)
0811 00:17:38.225385 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:38.225466 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:38953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 00:17:38.286110 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:38.285549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38953 (local address 127.0.0.1:48652)
0811 00:17:38.285774 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:38.285887 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:38953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 00:17:38.287452 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:38.287067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38953 (local address 127.0.0.1:48660)
0811 00:17:38.287227 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:38.287308 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:38953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 00:17:38.287850 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 8534
I20260811 00:17:38.315428 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 8753
I20260811 00:17:38.322722 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:38.316219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55686 (local address 127.7.94.3:44709)
0811 00:17:38.316415 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:38.316421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:38.316521 (+ 100us) server_negotiation.cc:407] Connection header received
0811 00:17:38.316735 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:38.316745 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:38.316833 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:38.316942 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:38.317426 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:38.318157 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:38.319489 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:38.319882 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:38.320038 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:38.321369 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:38.321380 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:38.321394 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:38.321934 (+ 540us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:38.321980 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:38.322005 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:38.322132 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:38.322271 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 00:17:38.322375 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:17:38.323341 8976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:38.368022 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:38.367498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38953 (local address 127.0.0.1:48670)
0811 00:17:38.367728 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:38.367852 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:38953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 00:17:38.369124 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:38.368710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38953 (local address 127.0.0.1:48686)
0811 00:17:38.368910 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:38.368977 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:38953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 00:17:38.411019 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 8753
I20260811 00:17:38.436904 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 8134
I20260811 00:17:38.438134 8240 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:38.545356 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 8134
I20260811 00:17:38.545390 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:38.544780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:38953 (local address 127.0.0.1:48702)
0811 00:17:38.544993 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:38.545175 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:38953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
2026-08-11T00:17:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2962 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:17:38 dist-test-slave-8hhm krb5kdc[9062](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:17:38 dist-test-slave-8hhm krb5kdc[9062](info): set up 2 sockets
Aug 11 00:17:38 dist-test-slave-8hhm krb5kdc[9062](info): commencing operation
krb5kdc: starting...
W20260811 00:17:39.931056 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.335s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:17:40 dist-test-slave-8hhm krb5kdc[9062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:17:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:17:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:17:40.228984 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:40409
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:40409
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:17:40.564826 9084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:40.565172 9084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:40.565361 9084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:40.572393 9084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:17:40.572530 9084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:40.572644 9084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:40.572751 9084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:17:40.572836 9084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:17:40.582201 9084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:40409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:40409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:40.583465 9084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:40.584798 9084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:40.596711 9089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:40.596736 9091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:40.596675 9093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:40.597558 9084 server_base.cc:1034] running on GCE node
I20260811 00:17:40.598235 9084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:40.599574 9084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:40.600929 9084 hybrid_clock.cc:648] HybridClock initialized: now 1786407460600870 us; error 46 us; skew 500 ppm
I20260811 00:17:40.601382 9084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:40 dist-test-slave-8hhm krb5kdc[9062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407460, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:40.604410 9084 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:17:40.605917 9084 webserver.cc:460] Webserver started at http://127.7.94.62:34939/ using document root <none> and password file <none>
I20260811 00:17:40.606607 9084 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:40.606757 9084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:40.607056 9084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:40.610001 9084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "109f4499fd434ae19f8cb83dfbf84b60"
format_stamp: "Formatted at 2026-08-11 00:17:40 on dist-test-slave-8hhm"
server_key: "ed623a5fe0dd128efad8823abdd0ef90"
server_key_iv: "5960eb8e0411c24da1b67197a2e0467e"
server_key_version: "encryptionkey@0"
I20260811 00:17:40.610739 9084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "109f4499fd434ae19f8cb83dfbf84b60"
format_stamp: "Formatted at 2026-08-11 00:17:40 on dist-test-slave-8hhm"
server_key: "ed623a5fe0dd128efad8823abdd0ef90"
server_key_iv: "5960eb8e0411c24da1b67197a2e0467e"
server_key_version: "encryptionkey@0"
I20260811 00:17:40.615833 9084 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:17:40.619197 9102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:40.620395 9084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:17:40.620574 9084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "109f4499fd434ae19f8cb83dfbf84b60"
format_stamp: "Formatted at 2026-08-11 00:17:40 on dist-test-slave-8hhm"
server_key: "ed623a5fe0dd128efad8823abdd0ef90"
server_key_iv: "5960eb8e0411c24da1b67197a2e0467e"
server_key_version: "encryptionkey@0"
I20260811 00:17:40.620782 9084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:40.643141 9084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:40.647187 9084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:40.647475 9084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:40.666117 9084 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:40409
I20260811 00:17:40.666208 9187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:40409 every 8 connection(s)
I20260811 00:17:40.667550 9084 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:17:40.669445 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 9084
I20260811 00:17:40.669646 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:17:40.670199 7544 external_mini_cluster.cc:1428] Setting key c7481075caf738a4d0f2a81097fac5ba
I20260811 00:17:40.672299 9189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 00:17:40 dist-test-slave-8hhm krb5kdc[9062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:17:40.685887 9189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60: Bootstrap starting.
I20260811 00:17:40.689311 9189 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:40.690403 9189 log.cc:824] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:40.693804 9189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60: No bootstrap required, opened a new log
I20260811 00:17:40.694641 9197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:40.672111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44198 (local address 127.7.94.62:40409)
0811 00:17:40.672915 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:40.672931 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:40.672982 (+ 51us) server_negotiation.cc:407] Connection header received
0811 00:17:40.675299 (+ 2317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:40.675325 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:40.675828 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:40.676259 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:40.677748 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:40.679809 (+ 2061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:40.681101 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:40.682284 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:40.682506 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:40.687076 (+ 4570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:40.687084 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:40.687110 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:40.687149 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:40.690365 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:40.691255 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:40.691269 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:40.691278 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:40.691496 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:40.691937 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:40.691941 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:40.691943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:40.692422 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:40.692651 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:40.693147 (+ 496us) server_negotiation.cc:299] Negotiation successful
0811 00:17:40.693535 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":360,"threads_started":1}
I20260811 00:17:40.701514 9189 raft_consensus.cc:348] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109f4499fd434ae19f8cb83dfbf84b60" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40409 } }
I20260811 00:17:40.701943 9189 raft_consensus.cc:374] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:40.702068 9189 raft_consensus.cc:729] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 109f4499fd434ae19f8cb83dfbf84b60, State: Initialized, Role: FOLLOWER
I20260811 00:17:40.702744 9189 consensus_queue.cc:260] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109f4499fd434ae19f8cb83dfbf84b60" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40409 } }
I20260811 00:17:40.703032 9189 raft_consensus.cc:388] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:17:40.703199 9189 raft_consensus.cc:482] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:17:40.703388 9189 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:40.705649 9189 raft_consensus.cc:504] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109f4499fd434ae19f8cb83dfbf84b60" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40409 } }
I20260811 00:17:40.706139 9189 leader_election.cc:302] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 109f4499fd434ae19f8cb83dfbf84b60; no voters:
I20260811 00:17:40.706700 9189 leader_election.cc:288] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:17:40.706833 9202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:17:40.707552 9202 raft_consensus.cc:686] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60 [term 1 LEADER]: Becoming Leader. State: Replica: 109f4499fd434ae19f8cb83dfbf84b60, State: Running, Role: LEADER
I20260811 00:17:40.708098 9202 consensus_queue.cc:237] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109f4499fd434ae19f8cb83dfbf84b60" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40409 } }
I20260811 00:17:40.709003 9189 sys_catalog.cc:564] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:17:40.711603 9212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 109f4499fd434ae19f8cb83dfbf84b60. Latest consensus state: current_term: 1 leader_uuid: "109f4499fd434ae19f8cb83dfbf84b60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109f4499fd434ae19f8cb83dfbf84b60" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40409 } } }
I20260811 00:17:40.711622 9205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "109f4499fd434ae19f8cb83dfbf84b60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109f4499fd434ae19f8cb83dfbf84b60" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40409 } } }
I20260811 00:17:40.712404 9212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60 [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:40.712563 9205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60 [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:40.714049 9219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:17:40.719471 9219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:17:40.727727 9219 catalog_manager.cc:1347] Generated new cluster ID: 60f51154a6c24deaa4c8d29a74950ac0
I20260811 00:17:40.727850 9219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:17:40.749094 9219 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:17:40.751016 9219 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:17:40.781373 9219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60: Generated new TSK 0
I20260811 00:17:40.782496 9219 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.94.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
Entry for principal kudu/127.7.94.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab.
I20260811 00:17:40.862776 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:40409
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:17:41.202754 9240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:41.203032 9240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:41.203126 9240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:41.210361 9240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:41.210454 9240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:41.210623 9240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:17:41.220291 9240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.1.keytab
--principal=kudu/127.7.94.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:40409
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:41.221498 9240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:41.222751 9240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:41.236239 9249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:41.236219 9257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:41.236277 9250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:41.237054 9240 server_base.cc:1034] running on GCE node
I20260811 00:17:41.237680 9240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:41.238582 9240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:41.239809 9240 hybrid_clock.cc:648] HybridClock initialized: now 1786407461239763 us; error 59 us; skew 500 ppm
I20260811 00:17:41.240118 9240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:41 dist-test-slave-8hhm krb5kdc[9062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407461, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:41.243335 9240 init.cc:377] Logged in from keytab as kudu/127.7.94.1@KRBTEST.COM (short username kudu)
I20260811 00:17:41.244809 9240 webserver.cc:460] Webserver started at http://127.7.94.1:41703/ using document root <none> and password file <none>
I20260811 00:17:41.245604 9240 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:41.245747 9240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:41.246044 9240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:41.248842 9240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "3c0426b891174c26b64e4a128abd8203"
format_stamp: "Formatted at 2026-08-11 00:17:41 on dist-test-slave-8hhm"
server_key: "2aa286c96d7ca207de2d6c04cf929c72"
server_key_iv: "6dfadbf6a459378a9a08ec7db8d4a86a"
server_key_version: "encryptionkey@0"
I20260811 00:17:41.249684 9240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "3c0426b891174c26b64e4a128abd8203"
format_stamp: "Formatted at 2026-08-11 00:17:41 on dist-test-slave-8hhm"
server_key: "2aa286c96d7ca207de2d6c04cf929c72"
server_key_iv: "6dfadbf6a459378a9a08ec7db8d4a86a"
server_key_version: "encryptionkey@0"
I20260811 00:17:41.254786 9240 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:17:41.258227 9270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:41.259433 9240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:17:41.259630 9240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "3c0426b891174c26b64e4a128abd8203"
format_stamp: "Formatted at 2026-08-11 00:17:41 on dist-test-slave-8hhm"
server_key: "2aa286c96d7ca207de2d6c04cf929c72"
server_key_iv: "6dfadbf6a459378a9a08ec7db8d4a86a"
server_key_version: "encryptionkey@0"
I20260811 00:17:41.259845 9240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:41.272248 9240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:41.276108 9240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:41.276429 9240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:41.277407 9240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:41.278733 9240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:17:41.278841 9240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:41.278986 9240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:41.279062 9240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:41.312862 9240 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:33541
I20260811 00:17:41.312909 9461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:33541 every 8 connection(s)
I20260811 00:17:41.314412 9240 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:17:41.324290 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 9240
I20260811 00:17:41.324481 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:17:41.324914 7544 external_mini_cluster.cc:1428] Setting key 0088ace34756882df407462ee5b8b658
Aug 11 00:17:41 dist-test-slave-8hhm krb5kdc[9062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407461, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.1@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:17:41.334558 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:41.317114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:40877 (local address 127.7.94.62:40409)
0811 00:17:41.317634 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:41.317639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:41.318485 (+ 846us) server_negotiation.cc:407] Connection header received
0811 00:17:41.320325 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:41.320330 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:41.320421 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:41.320542 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:41.322084 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:41.322823 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:41.324410 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:41.324995 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:41.325156 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:41.328926 (+ 3770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:41.328931 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:41.328934 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:41.328966 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:41.331537 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:41.332272 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:41.332277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:41.332279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:41.332402 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:41.332832 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:41.332836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:41.332844 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:41.333095 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:41.333279 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:41.333965 (+ 686us) server_negotiation.cc:299] Negotiation successful
0811 00:17:41.334192 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":306,"threads_started":1}
I20260811 00:17:41.335539 9467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:41.317598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40409 (local address 127.7.94.1:40877)
0811 00:17:41.318327 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:41.318365 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:41.320054 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:41.320786 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:41.320794 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:41.321352 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:17:41.321911 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:41.321929 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:41.322989 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:41.322994 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:41.324267 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:41.324280 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:41.325250 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:41.325260 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:41.325549 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:41.326296 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:17:41.326324 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:17:41.328717 (+ 2393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:17:41.331691 (+ 2974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:17:41.331713 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:17:41.331728 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:17:41.332139 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:17:41.332512 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:17:41.332517 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:17:41.332521 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:17:41.332713 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:17:41.333244 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:17:41.333252 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:17:41.333581 (+ 329us) client_negotiation.cc:769] Sending connection context
0811 00:17:41.333844 (+ 263us) client_negotiation.cc:240] Negotiation successful
0811 00:17:41.334368 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":316,"threads_started":1}
I20260811 00:17:41.337189 9463 heartbeater.cc:347] Connected to a master server at 127.7.94.62:40409
I20260811 00:17:41.337524 9463 heartbeater.cc:464] Registering TS with master...
I20260811 00:17:41.338240 9463 heartbeater.cc:511] Master 127.7.94.62:40409 requested a full tablet report, sending...
I20260811 00:17:41.340485 9129 ts_manager.cc:194] Registered new tserver with Master: 3c0426b891174c26b64e4a128abd8203 (127.7.94.1:33541)
I20260811 00:17:41.341988 9129 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.1@KRBTEST.COM'} at 127.7.94.1:40877
WARNING: no policy specified for kudu/127.7.94.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
Entry for principal kudu/127.7.94.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab.
I20260811 00:17:41.396150 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:40409
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:17:41.736575 9472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:41.736891 9472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:41.737071 9472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:41.743906 9472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:41.744051 9472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:41.744213 9472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:17:41.753764 9472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.2.keytab
--principal=kudu/127.7.94.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:40409
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:41.755033 9472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:41.756312 9472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:41.769388 9485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:41.769461 9481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:41.769482 9483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:41.770447 9472 server_base.cc:1034] running on GCE node
I20260811 00:17:41.771027 9472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:41.771952 9472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:41.773154 9472 hybrid_clock.cc:648] HybridClock initialized: now 1786407461773119 us; error 31 us; skew 500 ppm
I20260811 00:17:41.773504 9472 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:41 dist-test-slave-8hhm krb5kdc[9062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407461, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:41.776760 9472 init.cc:377] Logged in from keytab as kudu/127.7.94.2@KRBTEST.COM (short username kudu)
I20260811 00:17:41.778246 9472 webserver.cc:460] Webserver started at http://127.7.94.2:46545/ using document root <none> and password file <none>
I20260811 00:17:41.778952 9472 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:41.779088 9472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:41.779418 9472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:41.782337 9472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "256a2816052043a2a6faef8ad22dbe0f"
format_stamp: "Formatted at 2026-08-11 00:17:41 on dist-test-slave-8hhm"
server_key: "8cd247247b1ed044090a5d736af866bb"
server_key_iv: "292bdcc80952c591805ef17b34ad3e92"
server_key_version: "encryptionkey@0"
I20260811 00:17:41.783134 9472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "256a2816052043a2a6faef8ad22dbe0f"
format_stamp: "Formatted at 2026-08-11 00:17:41 on dist-test-slave-8hhm"
server_key: "8cd247247b1ed044090a5d736af866bb"
server_key_iv: "292bdcc80952c591805ef17b34ad3e92"
server_key_version: "encryptionkey@0"
I20260811 00:17:41.788152 9472 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 00:17:41.791610 9498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:41.792883 9472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:17:41.793099 9472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "256a2816052043a2a6faef8ad22dbe0f"
format_stamp: "Formatted at 2026-08-11 00:17:41 on dist-test-slave-8hhm"
server_key: "8cd247247b1ed044090a5d736af866bb"
server_key_iv: "292bdcc80952c591805ef17b34ad3e92"
server_key_version: "encryptionkey@0"
I20260811 00:17:41.793365 9472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:41.808187 9472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:41.812382 9472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:41.812667 9472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:41.813652 9472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:41.815089 9472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:17:41.815184 9472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:41.815310 9472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:41.815384 9472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:41.848287 9472 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:35005
I20260811 00:17:41.848327 9686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:35005 every 8 connection(s)
I20260811 00:17:41.849705 9472 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:17:41.857191 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 9472
I20260811 00:17:41.857429 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:17:41.857862 7544 external_mini_cluster.cc:1428] Setting key a6f86d0e5134fa6e2320775940d24c91
Aug 11 00:17:41 dist-test-slave-8hhm krb5kdc[9062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407461, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.2@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:17:41.870795 9691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:41.852295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:49179 (local address 127.7.94.62:40409)
0811 00:17:41.852749 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:41.852754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:41.853625 (+ 871us) server_negotiation.cc:407] Connection header received
0811 00:17:41.855434 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:41.855441 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:41.855530 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:41.855660 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:41.857124 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:41.858000 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:41.859687 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:41.860350 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:41.860511 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:41.864010 (+ 3499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:41.864019 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:41.864027 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:41.864073 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:41.867604 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:41.868279 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:41.868287 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:41.868293 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:41.868456 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:41.868858 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:41.868865 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:41.868871 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:41.869287 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:41.869438 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:41.870075 (+ 637us) server_negotiation.cc:299] Negotiation successful
0811 00:17:41.870319 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":245,"threads_started":1}
I20260811 00:17:41.871464 9692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:41.852692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40409 (local address 127.7.94.2:49179)
0811 00:17:41.853445 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:41.853488 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:41.855158 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:41.855906 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:41.855914 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:41.856399 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:17:41.856948 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:41.856966 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:41.858183 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:41.858187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:41.859547 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:41.859564 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:41.860569 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:41.860574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:41.860813 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:41.861502 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:17:41.861523 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:17:41.863786 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:17:41.867738 (+ 3952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:17:41.867744 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:17:41.867754 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:17:41.868146 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:17:41.868566 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:17:41.868569 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:17:41.868572 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:17:41.868751 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:17:41.869409 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:17:41.869414 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:17:41.869710 (+ 296us) client_negotiation.cc:769] Sending connection context
0811 00:17:41.869928 (+ 218us) client_negotiation.cc:240] Negotiation successful
0811 00:17:41.870365 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":279,"threads_started":1}
I20260811 00:17:41.872882 9689 heartbeater.cc:347] Connected to a master server at 127.7.94.62:40409
I20260811 00:17:41.873231 9689 heartbeater.cc:464] Registering TS with master...
I20260811 00:17:41.874084 9689 heartbeater.cc:511] Master 127.7.94.62:40409 requested a full tablet report, sending...
I20260811 00:17:41.875702 9129 ts_manager.cc:194] Registered new tserver with Master: 256a2816052043a2a6faef8ad22dbe0f (127.7.94.2:35005)
I20260811 00:17:41.876915 9129 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.2@KRBTEST.COM'} at 127.7.94.2:49179
WARNING: no policy specified for kudu/127.7.94.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
Entry for principal kudu/127.7.94.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab.
I20260811 00:17:41.927959 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:40409
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:17:42.260979 9699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:42.261297 9699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:42.261389 9699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:42.268395 9699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:42.268498 9699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:42.268607 9699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:17:42.278304 9699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.3.keytab
--principal=kudu/127.7.94.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:40409
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:42.279556 9699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:42.280886 9699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:42.294040 9706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:42.294077 9711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:42.294116 9707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:42.294991 9699 server_base.cc:1034] running on GCE node
I20260811 00:17:42.295668 9699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:42.296657 9699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:42.297899 9699 hybrid_clock.cc:648] HybridClock initialized: now 1786407462297830 us; error 57 us; skew 500 ppm
I20260811 00:17:42.298215 9699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:42 dist-test-slave-8hhm krb5kdc[9062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407462, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:42.301460 9699 init.cc:377] Logged in from keytab as kudu/127.7.94.3@KRBTEST.COM (short username kudu)
I20260811 00:17:42.302863 9699 webserver.cc:460] Webserver started at http://127.7.94.3:38639/ using document root <none> and password file <none>
I20260811 00:17:42.303520 9699 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:42.303627 9699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:42.303891 9699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:42.306749 9699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "9b3b731044554dbcba2a47e11fe81744"
format_stamp: "Formatted at 2026-08-11 00:17:42 on dist-test-slave-8hhm"
server_key: "6cf57baf5d3b386de7073b0ef5a6dac2"
server_key_iv: "64343cd5fecea0252eb3723202ebccf1"
server_key_version: "encryptionkey@0"
I20260811 00:17:42.307489 9699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "9b3b731044554dbcba2a47e11fe81744"
format_stamp: "Formatted at 2026-08-11 00:17:42 on dist-test-slave-8hhm"
server_key: "6cf57baf5d3b386de7073b0ef5a6dac2"
server_key_iv: "64343cd5fecea0252eb3723202ebccf1"
server_key_version: "encryptionkey@0"
I20260811 00:17:42.312793 9699 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 00:17:42.317081 9722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:42.318647 9699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 00:17:42.318822 9699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "9b3b731044554dbcba2a47e11fe81744"
format_stamp: "Formatted at 2026-08-11 00:17:42 on dist-test-slave-8hhm"
server_key: "6cf57baf5d3b386de7073b0ef5a6dac2"
server_key_iv: "64343cd5fecea0252eb3723202ebccf1"
server_key_version: "encryptionkey@0"
I20260811 00:17:42.319023 9699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:42.334125 9699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:42.338172 9699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:42.338454 9699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:42.339334 9699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:42.340544 9699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:17:42.340619 9699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:42.340791 9699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:42.340842 9699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:42.345494 9463 heartbeater.cc:503] Master 127.7.94.62:40409 was elected leader, sending a full tablet report...
I20260811 00:17:42.375025 9699 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:33093
I20260811 00:17:42.375116 9902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:33093 every 8 connection(s)
I20260811 00:17:42.376299 9699 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:17:42.386482 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 9699
I20260811 00:17:42.386682 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:17:42.387111 7544 external_mini_cluster.cc:1428] Setting key 46df518577111247cd2d1124df8cf0e8
Aug 11 00:17:42 dist-test-slave-8hhm krb5kdc[9062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407462, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.3@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:17:42.395921 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:42.378784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:51379 (local address 127.7.94.62:40409)
0811 00:17:42.379264 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:42.379269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:42.380225 (+ 956us) server_negotiation.cc:407] Connection header received
0811 00:17:42.382046 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:42.382050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:42.382111 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:42.382211 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:42.383611 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.384363 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.385970 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.386551 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.386686 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:42.390345 (+ 3659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:42.390351 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:42.390358 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:42.390425 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:42.393273 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:42.393924 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:42.393927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:42.393929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:42.394047 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:42.394451 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:42.394456 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:42.394459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:42.394745 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:42.394888 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:42.395471 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 00:17:42.395633 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":251,"threads_started":1}
I20260811 00:17:42.396899 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:42.379316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:40409 (local address 127.7.94.3:51379)
0811 00:17:42.380041 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:42.380080 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:42.381813 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:42.382411 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:42.382419 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:42.382921 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:17:42.383464 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:42.383481 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.384500 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.384504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:42.385843 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:42.385856 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.386816 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.386824 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:42.387020 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:42.387763 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:17:42.387785 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:17:42.390135 (+ 2350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:17:42.393394 (+ 3259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:17:42.393400 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:17:42.393410 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:17:42.393818 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:17:42.394139 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:17:42.394142 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:17:42.394145 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:17:42.394313 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:17:42.394862 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:17:42.394881 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:17:42.395159 (+ 278us) client_negotiation.cc:769] Sending connection context
0811 00:17:42.395368 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 00:17:42.395777 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":293,"threads_started":1}
I20260811 00:17:42.398275 9904 heartbeater.cc:347] Connected to a master server at 127.7.94.62:40409
I20260811 00:17:42.398589 9904 heartbeater.cc:464] Registering TS with master...
I20260811 00:17:42.399288 9904 heartbeater.cc:511] Master 127.7.94.62:40409 requested a full tablet report, sending...
I20260811 00:17:42.400556 9129 ts_manager.cc:194] Registered new tserver with Master: 9b3b731044554dbcba2a47e11fe81744 (127.7.94.3:33093)
I20260811 00:17:42.401652 9129 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.94.3@KRBTEST.COM'} at 127.7.94.3:51379
I20260811 00:17:42.402853 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:17:42.420957 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:42.407027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41372 (local address 127.7.94.62:40409)
0811 00:17:42.407265 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:42.407271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:42.407552 (+ 281us) server_negotiation.cc:407] Connection header received
0811 00:17:42.407681 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:42.407685 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:42.407756 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:42.407841 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:42.409637 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.410559 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.411870 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.412437 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.412533 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:42.415819 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:42.415823 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:42.415827 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:42.415860 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:42.418487 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:42.419120 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:42.419124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:42.419127 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:42.419241 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:42.419567 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:42.419584 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:42.419586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:42.419889 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:42.420095 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:42.420424 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 00:17:42.420610 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 00:17:42.425434 9129 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41372:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:17:42.433063 9129 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:17:42.467903 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:42.447373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39046 (local address 127.7.94.1:33541)
0811 00:17:42.459035 (+ 11662us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:42.459048 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:42.459086 (+ 38us) server_negotiation.cc:407] Connection header received
0811 00:17:42.459242 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:42.459253 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:42.459603 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:42.459843 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:42.461020 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.462691 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.465346 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.466751 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.466883 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:42.467136 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:42.467395 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 00:17:42.467558 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11435,"thread_start_us":12114,"threads_started":1}
I20260811 00:17:42.468034 9927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:42.446607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:33541 (local address 127.0.0.1:39046)
0811 00:17:42.453558 (+ 6951us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:42.453582 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:42.453739 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:42.460131 (+ 6392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:42.460138 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:42.460191 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:42.460765 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:42.460778 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.463075 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.463080 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:42.465129 (+ 2049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:42.465142 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.466933 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.466939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:42.467140 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:42.467209 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 00:17:42.467351 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 00:17:42.467541 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6779,"mutex_wait_us":103,"spinlock_wait_cycles":2816,"thread_start_us":996,"threads_started":1}
I20260811 00:17:42.468536 9923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:42.445461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:35005 (local address 127.0.0.1:37950)
0811 00:17:42.446206 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:42.446243 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:42.446409 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:42.460141 (+ 13732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:42.460144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:42.460183 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:42.460685 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:42.460704 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.462893 (+ 2189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.462898 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:42.465870 (+ 2972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:42.465885 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.467657 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.467664 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:42.467835 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:42.467872 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 00:17:42.467948 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 00:17:42.468077 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":94,"spinlock_wait_cycles":768,"thread_start_us":375,"threads_started":1}
I20260811 00:17:42.468819 9928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:42.447235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:33093 (local address 127.0.0.1:59444)
0811 00:17:42.458197 (+ 10962us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:42.458221 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:42.458363 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:42.460698 (+ 2335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:42.460703 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:42.460731 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:42.461109 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:42.461119 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.463019 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.463022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:42.465870 (+ 2848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:42.465885 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.467958 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.467962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:42.468112 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:42.468138 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 00:17:42.468227 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 00:17:42.468339 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10773,"mutex_wait_us":199,"thread_start_us":10595,"threads_started":1}
I20260811 00:17:42.469190 9932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:42.447546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59444 (local address 127.7.94.3:33093)
0811 00:17:42.460194 (+ 12648us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:42.460203 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:42.460234 (+ 31us) server_negotiation.cc:407] Connection header received
0811 00:17:42.460330 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:42.460337 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:42.460511 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:42.460702 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:42.461321 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.462889 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.466057 (+ 3168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.467428 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.467564 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:42.467811 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:42.468648 (+ 837us) server_negotiation.cc:299] Negotiation successful
0811 00:17:42.468852 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12449,"thread_start_us":12280,"threads_started":1}
I20260811 00:17:42.469774 9925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:42.445498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37950 (local address 127.7.94.2:35005)
0811 00:17:42.459522 (+ 14024us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:42.459532 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:42.459563 (+ 31us) server_negotiation.cc:407] Connection header received
0811 00:17:42.459679 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:42.459688 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:42.459961 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:42.460171 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:42.460933 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.462691 (+ 1758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.466057 (+ 3366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.467443 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.469009 (+ 1566us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:42.469237 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:42.469319 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 00:17:42.469504 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13872,"thread_start_us":13758,"threads_started":1}
I20260811 00:17:42.472404 9351 tablet_service.cc:1467] Processing CreateTablet for tablet 039be629e51041cabac045cb71472411 (DEFAULT_TABLE table=test-table [id=8384b452f35c4f358775144b76c16ead]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:17:42.472831 9793 tablet_service.cc:1467] Processing CreateTablet for tablet 039be629e51041cabac045cb71472411 (DEFAULT_TABLE table=test-table [id=8384b452f35c4f358775144b76c16ead]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:17:42.474099 9351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 039be629e51041cabac045cb71472411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:42.474362 9793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 039be629e51041cabac045cb71472411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:42.475210 9574 tablet_service.cc:1467] Processing CreateTablet for tablet 039be629e51041cabac045cb71472411 (DEFAULT_TABLE table=test-table [id=8384b452f35c4f358775144b76c16ead]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:17:42.476214 9574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 039be629e51041cabac045cb71472411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:42.486783 9935 tablet_bootstrap.cc:492] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f: Bootstrap starting.
I20260811 00:17:42.489586 9937 tablet_bootstrap.cc:492] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203: Bootstrap starting.
I20260811 00:17:42.490355 9938 tablet_bootstrap.cc:492] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744: Bootstrap starting.
I20260811 00:17:42.491605 9935 tablet_bootstrap.cc:654] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:42.492988 9935 log.cc:824] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:42.493963 9938 tablet_bootstrap.cc:654] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:42.495146 9938 log.cc:824] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:42.496326 9935 tablet_bootstrap.cc:492] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f: No bootstrap required, opened a new log
I20260811 00:17:42.496519 9937 tablet_bootstrap.cc:654] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:42.496803 9935 ts_tablet_manager.cc:1397] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 00:17:42.497897 9937 log.cc:824] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:42.498425 9938 tablet_bootstrap.cc:492] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744: No bootstrap required, opened a new log
I20260811 00:17:42.498694 9938 ts_tablet_manager.cc:1397] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 00:17:42.500375 9937 tablet_bootstrap.cc:492] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203: No bootstrap required, opened a new log
I20260811 00:17:42.500635 9937 ts_tablet_manager.cc:1397] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260811 00:17:42.507477 9935 raft_consensus.cc:348] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256a2816052043a2a6faef8ad22dbe0f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35005 } } peers { permanent_uuid: "3c0426b891174c26b64e4a128abd8203" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33541 } } peers { permanent_uuid: "9b3b731044554dbcba2a47e11fe81744" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33093 } }
I20260811 00:17:42.507963 9935 raft_consensus.cc:374] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:42.508050 9935 raft_consensus.cc:729] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256a2816052043a2a6faef8ad22dbe0f, State: Initialized, Role: FOLLOWER
I20260811 00:17:42.508075 9937 raft_consensus.cc:348] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256a2816052043a2a6faef8ad22dbe0f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35005 } } peers { permanent_uuid: "3c0426b891174c26b64e4a128abd8203" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33541 } } peers { permanent_uuid: "9b3b731044554dbcba2a47e11fe81744" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33093 } }
I20260811 00:17:42.508472 9937 raft_consensus.cc:374] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:42.508569 9937 raft_consensus.cc:729] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c0426b891174c26b64e4a128abd8203, State: Initialized, Role: FOLLOWER
I20260811 00:17:42.508777 9935 consensus_queue.cc:260] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256a2816052043a2a6faef8ad22dbe0f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35005 } } peers { permanent_uuid: "3c0426b891174c26b64e4a128abd8203" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33541 } } peers { permanent_uuid: "9b3b731044554dbcba2a47e11fe81744" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33093 } }
I20260811 00:17:42.509172 9937 consensus_queue.cc:260] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256a2816052043a2a6faef8ad22dbe0f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35005 } } peers { permanent_uuid: "3c0426b891174c26b64e4a128abd8203" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33541 } } peers { permanent_uuid: "9b3b731044554dbcba2a47e11fe81744" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33093 } }
I20260811 00:17:42.509099 9938 raft_consensus.cc:348] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256a2816052043a2a6faef8ad22dbe0f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35005 } } peers { permanent_uuid: "3c0426b891174c26b64e4a128abd8203" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33541 } } peers { permanent_uuid: "9b3b731044554dbcba2a47e11fe81744" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33093 } }
I20260811 00:17:42.509598 9938 raft_consensus.cc:374] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:42.509717 9938 raft_consensus.cc:729] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b3b731044554dbcba2a47e11fe81744, State: Initialized, Role: FOLLOWER
I20260811 00:17:42.510214 9689 heartbeater.cc:503] Master 127.7.94.62:40409 was elected leader, sending a full tablet report...
I20260811 00:17:42.510440 9938 consensus_queue.cc:260] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256a2816052043a2a6faef8ad22dbe0f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35005 } } peers { permanent_uuid: "3c0426b891174c26b64e4a128abd8203" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33541 } } peers { permanent_uuid: "9b3b731044554dbcba2a47e11fe81744" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33093 } }
I20260811 00:17:42.511590 9904 heartbeater.cc:503] Master 127.7.94.62:40409 was elected leader, sending a full tablet report...
I20260811 00:17:42.512215 9935 ts_tablet_manager.cc:1428] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260811 00:17:42.512219 9938 ts_tablet_manager.cc:1428] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744: Time spent starting tablet: real 0.013s user 0.008s sys 0.005s
I20260811 00:17:42.513684 9937 ts_tablet_manager.cc:1428] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
W20260811 00:17:42.566947 9464 tablet.cc:2369] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:17:42.601967 9690 tablet.cc:2369] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:17:42.607121 9945 raft_consensus.cc:482] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:17:42.607430 9945 raft_consensus.cc:504] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256a2816052043a2a6faef8ad22dbe0f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35005 } } peers { permanent_uuid: "3c0426b891174c26b64e4a128abd8203" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33541 } } peers { permanent_uuid: "9b3b731044554dbcba2a47e11fe81744" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33093 } }
I20260811 00:17:42.609117 9945 leader_election.cc:288] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c0426b891174c26b64e4a128abd8203 (127.7.94.1:33541), 9b3b731044554dbcba2a47e11fe81744 (127.7.94.3:33093)
I20260811 00:17:42.617208 9932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:42.609623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:54257 (local address 127.7.94.3:33093)
0811 00:17:42.609824 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:42.609830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:42.610608 (+ 778us) server_negotiation.cc:407] Connection header received
0811 00:17:42.610919 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:42.610926 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:42.611002 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:42.611095 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:42.611788 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.612605 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.615448 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.616199 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.616349 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:42.616461 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:42.616816 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 00:17:42.616956 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 00:17:42.617264 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:42.609486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:38707 (local address 127.7.94.1:33541)
0811 00:17:42.609699 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:42.609707 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:42.610957 (+ 1250us) server_negotiation.cc:407] Connection header received
0811 00:17:42.611128 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:42.611133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:42.611214 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:42.611312 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:42.612003 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.612876 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.615487 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.616253 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.616414 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:42.616521 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:42.616855 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 00:17:42.616981 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 00:17:42.617457 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:42.609839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:33541 (local address 127.7.94.2:38707)
0811 00:17:42.610848 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:42.610865 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:42.610984 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:42.611340 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:42.611343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:42.611361 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:42.611816 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:42.611830 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.613006 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.613013 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:42.615284 (+ 2271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:42.615298 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.616485 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.616491 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:42.616676 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:42.616698 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 00:17:42.616796 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 00:17:42.616936 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"mutex_wait_us":80,"thread_start_us":687,"threads_started":1}
I20260811 00:17:42.617744 9949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:42.609618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:33093 (local address 127.7.94.2:54257)
0811 00:17:42.610497 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:42.610519 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:42.610733 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:42.611122 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:42.611126 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:42.611152 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:42.611592 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:42.611610 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.612737 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.612745 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:42.615285 (+ 2540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:42.615298 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.616440 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.616448 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:42.616617 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:42.616672 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 00:17:42.616776 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 00:17:42.616925 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"mutex_wait_us":178,"thread_start_us":470,"threads_started":1}
I20260811 00:17:42.618536 9389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039be629e51041cabac045cb71472411" candidate_uuid: "256a2816052043a2a6faef8ad22dbe0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c0426b891174c26b64e4a128abd8203" is_pre_election: true
I20260811 00:17:42.618726 9830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039be629e51041cabac045cb71472411" candidate_uuid: "256a2816052043a2a6faef8ad22dbe0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b3b731044554dbcba2a47e11fe81744" is_pre_election: true
I20260811 00:17:42.619058 9389 raft_consensus.cc:2454] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 256a2816052043a2a6faef8ad22dbe0f in term 0.
I20260811 00:17:42.619246 9830 raft_consensus.cc:2454] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 256a2816052043a2a6faef8ad22dbe0f in term 0.
I20260811 00:17:42.619758 9500 leader_election.cc:302] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 256a2816052043a2a6faef8ad22dbe0f, 3c0426b891174c26b64e4a128abd8203; no voters:
I20260811 00:17:42.620193 9945 raft_consensus.cc:2781] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:17:42.620344 9945 raft_consensus.cc:482] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:17:42.620460 9945 raft_consensus.cc:3037] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:42.622771 9945 raft_consensus.cc:504] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256a2816052043a2a6faef8ad22dbe0f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35005 } } peers { permanent_uuid: "3c0426b891174c26b64e4a128abd8203" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33541 } } peers { permanent_uuid: "9b3b731044554dbcba2a47e11fe81744" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33093 } }
I20260811 00:17:42.623410 9945 leader_election.cc:288] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [CANDIDATE]: Term 1 election: Requested vote from peers 3c0426b891174c26b64e4a128abd8203 (127.7.94.1:33541), 9b3b731044554dbcba2a47e11fe81744 (127.7.94.3:33093)
I20260811 00:17:42.623817 9389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039be629e51041cabac045cb71472411" candidate_uuid: "256a2816052043a2a6faef8ad22dbe0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c0426b891174c26b64e4a128abd8203"
I20260811 00:17:42.623944 9830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "039be629e51041cabac045cb71472411" candidate_uuid: "256a2816052043a2a6faef8ad22dbe0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b3b731044554dbcba2a47e11fe81744"
I20260811 00:17:42.624006 9389 raft_consensus.cc:3037] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:42.624166 9830 raft_consensus.cc:3037] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:42.626382 9389 raft_consensus.cc:2454] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 256a2816052043a2a6faef8ad22dbe0f in term 1.
I20260811 00:17:42.626577 9830 raft_consensus.cc:2454] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 256a2816052043a2a6faef8ad22dbe0f in term 1.
I20260811 00:17:42.626796 9500 leader_election.cc:302] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 256a2816052043a2a6faef8ad22dbe0f, 3c0426b891174c26b64e4a128abd8203; no voters:
I20260811 00:17:42.627056 9945 raft_consensus.cc:2781] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:17:42.627492 9945 raft_consensus.cc:686] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [term 1 LEADER]: Becoming Leader. State: Replica: 256a2816052043a2a6faef8ad22dbe0f, State: Running, Role: LEADER
W20260811 00:17:42.627892 9906 tablet.cc:2369] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:17:42.628046 9945 consensus_queue.cc:237] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256a2816052043a2a6faef8ad22dbe0f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35005 } } peers { permanent_uuid: "3c0426b891174c26b64e4a128abd8203" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33541 } } peers { permanent_uuid: "9b3b731044554dbcba2a47e11fe81744" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33093 } }
I20260811 00:17:42.632409 9126 catalog_manager.cc:5495] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 256a2816052043a2a6faef8ad22dbe0f (127.7.94.2). New cstate: current_term: 1 leader_uuid: "256a2816052043a2a6faef8ad22dbe0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "256a2816052043a2a6faef8ad22dbe0f" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 35005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c0426b891174c26b64e4a128abd8203" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b3b731044554dbcba2a47e11fe81744" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33093 } health_report { overall_health: UNKNOWN } } }
I20260811 00:17:42.657680 9925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:42.653015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37960 (local address 127.7.94.2:35005)
0811 00:17:42.653201 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:42.653205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:42.653257 (+ 52us) server_negotiation.cc:407] Connection header received
0811 00:17:42.653339 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:42.653342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:42.653412 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:42.653534 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:17:42.653867 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.654550 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.655807 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.656190 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.656293 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:42.656717 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:17:42.657183 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:17:42.657367 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:42.657409 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 00:17:42.657468 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 00:17:42.665465 9389 raft_consensus.cc:1261] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203 [term 1 FOLLOWER]: Refusing update from remote peer 256a2816052043a2a6faef8ad22dbe0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:17:42.665500 9830 raft_consensus.cc:1261] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744 [term 1 FOLLOWER]: Refusing update from remote peer 256a2816052043a2a6faef8ad22dbe0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:17:42.666453 9951 consensus_queue.cc:1035] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c0426b891174c26b64e4a128abd8203" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:17:42.666836 9945 consensus_queue.cc:1035] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b3b731044554dbcba2a47e11fe81744" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 33093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:17:42.684592 9958 mvcc.cc:205] Tried to move back new op lower bound from 7317124967068086272 to 7317124966928809984. Current Snapshot: MvccSnapshot[applied={T|T < 7317124967068086272}]
I20260811 00:17:42.704188 9126 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:17:42.704478 9126 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:17:42.704818 9960 mvcc.cc:205] Tried to move back new op lower bound from 7317124967068086272 to 7317124966928809984. Current Snapshot: MvccSnapshot[applied={T|T < 7317124967068086272}]
I20260811 00:17:42.708747 9126 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 109f4499fd434ae19f8cb83dfbf84b60: Sending DeleteTablet for 3 replicas of tablet 039be629e51041cabac045cb71472411
I20260811 00:17:42.709988 9351 tablet_service.cc:1514] Processing DeleteTablet for tablet 039be629e51041cabac045cb71472411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:17:42 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:39046
I20260811 00:17:42.710534 9574 tablet_service.cc:1514] Processing DeleteTablet for tablet 039be629e51041cabac045cb71472411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:17:42 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:37950
I20260811 00:17:42.710724 9793 tablet_service.cc:1514] Processing DeleteTablet for tablet 039be629e51041cabac045cb71472411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:17:42 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:59444
I20260811 00:17:42.710862 9970 tablet_replica.cc:330] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203: stopping tablet replica
I20260811 00:17:42.711272 9970 raft_consensus.cc:2229] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:17:42.711298 9350 tablet_service.cc:1514] Processing DeleteTablet for tablet 039be629e51041cabac045cb71472411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:17:42 UTC) from {username='kudu', principal='kudu/127.7.94.62@KRBTEST.COM'} at 127.0.0.1:39046
I20260811 00:17:42.711782 9970 raft_consensus.cc:2258] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:17:42.712077 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 9240
I20260811 00:17:42.721074 9970 ts_tablet_manager.cc:1905] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:17:42.723078 9974 tablet_replica.cc:330] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f: stopping tablet replica
I20260811 00:17:42.723644 9974 raft_consensus.cc:2229] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:17:42.724463 9974 raft_consensus.cc:2258] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:17:42.725674 9975 tablet_replica.cc:330] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744: stopping tablet replica
I20260811 00:17:42.726071 9975 raft_consensus.cc:2229] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:17:42.726549 9975 raft_consensus.cc:2258] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:17:42.727530 9974 ts_tablet_manager.cc:1905] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:17:42.728554 9975 ts_tablet_manager.cc:1905] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:17:42.731223 9970 ts_tablet_manager.cc:1918] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:17:42.731383 9970 log.cc:1196] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-0/wal/wals/039be629e51041cabac045cb71472411
I20260811 00:17:42.732036 9970 ts_tablet_manager.cc:1939] T 039be629e51041cabac045cb71472411 P 3c0426b891174c26b64e4a128abd8203: Deleting consensus metadata
I20260811 00:17:42.733994 9103 catalog_manager.cc:4841] TS 3c0426b891174c26b64e4a128abd8203 (127.7.94.1:33541): tablet 039be629e51041cabac045cb71472411 (table test-table [id=8384b452f35c4f358775144b76c16ead]) successfully deleted
I20260811 00:17:42.735050 9974 ts_tablet_manager.cc:1918] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260811 00:17:42.735070 9103 catalog_manager.cc:4808] TS 3c0426b891174c26b64e4a128abd8203 (127.7.94.1:33541): delete failed for tablet 039be629e51041cabac045cb71472411 because the tablet was not found. No further retry: Not found: Tablet not found: 039be629e51041cabac045cb71472411
I20260811 00:17:42.735275 9975 ts_tablet_manager.cc:1918] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:17:42.735441 9975 log.cc:1196] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/039be629e51041cabac045cb71472411
I20260811 00:17:42.735425 9974 log.cc:1196] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/039be629e51041cabac045cb71472411
Aug 11 00:17:42 dist-test-slave-8hhm krb5kdc[9062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.1@KRBTEST.COM
I20260811 00:17:42.735962 9975 ts_tablet_manager.cc:1939] T 039be629e51041cabac045cb71472411 P 9b3b731044554dbcba2a47e11fe81744: Deleting consensus metadata
I20260811 00:17:42.735962 9974 ts_tablet_manager.cc:1939] T 039be629e51041cabac045cb71472411 P 256a2816052043a2a6faef8ad22dbe0f: Deleting consensus metadata
I20260811 00:17:42.737519 9103 catalog_manager.cc:4841] TS 256a2816052043a2a6faef8ad22dbe0f (127.7.94.2:35005): tablet 039be629e51041cabac045cb71472411 (table test-table [id=8384b452f35c4f358775144b76c16ead]) successfully deleted
I20260811 00:17:42.737496 9105 catalog_manager.cc:4841] TS 9b3b731044554dbcba2a47e11fe81744 (127.7.94.3:33093): tablet 039be629e51041cabac045cb71472411 (table test-table [id=8384b452f35c4f358775144b76c16ead]) successfully deleted
I20260811 00:17:42.742571 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:42.713122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39054 (local address 127.7.94.1:33541)
0811 00:17:42.713353 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:42.713362 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:42.725085 (+ 11723us) server_negotiation.cc:407] Connection header received
0811 00:17:42.725329 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:42.725334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:42.725422 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:42.725527 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:42.727344 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.728441 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.730348 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.730996 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.731140 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:42.736639 (+ 5499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:42.736650 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:42.736662 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:42.736697 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:42.739685 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:42.740405 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:42.740413 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:42.740415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:42.740546 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:42.740945 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:42.740949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:42.740951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:42.741376 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:42.741724 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:42.742008 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 00:17:42.742221 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 00:17:42.743284 9456 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:42.834419 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 9240
I20260811 00:17:42.860512 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 9472
Aug 11 00:17:42 dist-test-slave-8hhm krb5kdc[9062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.2@KRBTEST.COM
I20260811 00:17:42.881543 9925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:42.861423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37964 (local address 127.7.94.2:35005)
0811 00:17:42.861634 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:42.861643 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:42.861720 (+ 77us) server_negotiation.cc:407] Connection header received
0811 00:17:42.861901 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:42.861909 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:42.862003 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:42.862127 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:42.863864 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.865265 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.867232 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:42.867979 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:42.868143 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:42.873809 (+ 5666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:42.873822 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:42.873842 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:42.873888 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:42.878009 (+ 4121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:42.878886 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:42.878896 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:42.878900 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:42.879099 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:42.879522 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:42.879528 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:42.879531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:42.880008 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:42.880361 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:42.880585 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 00:17:42.880833 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:17:42.882532 9679 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:43.000201 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 9472
I20260811 00:17:43.034461 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 9699
Aug 11 00:17:43 dist-test-slave-8hhm krb5kdc[9062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.3@KRBTEST.COM
I20260811 00:17:43.049064 9932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:43.035263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59456 (local address 127.7.94.3:33093)
0811 00:17:43.035436 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:43.035441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:43.035459 (+ 18us) server_negotiation.cc:407] Connection header received
0811 00:17:43.035608 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:43.035612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:43.035677 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:43.035758 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:43.036985 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:43.037732 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:43.038903 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:43.039257 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:43.039341 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:43.043674 (+ 4333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:43.043687 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:43.043704 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:43.043742 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:43.046576 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:43.047212 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:43.047219 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:43.047222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:43.047369 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:43.047669 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:43.047673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:43.047675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:43.048014 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:43.048293 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:43.048542 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 00:17:43.048709 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 00:17:43.049736 9896 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:43.137333 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 9699
I20260811 00:17:43.158839 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 9084
I20260811 00:17:43.159904 9180 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:43.269170 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 9084
2026-08-11T00:17:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4726 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T00:17:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:17:43Z Disabled control of system clock
I20260811 00:17:43.368261 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:45395
--webserver_interface=127.7.94.62
--webserver_port=0
--builtin_ntp_servers=127.7.94.20:42115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:45395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 00:17:43.705293 9986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:43.705581 9986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:43.705744 9986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:43.712881 9986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:17:43.713001 9986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:43.713095 9986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:43.713239 9986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:17:43.713336 9986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:17:43.722553 9986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:42115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:45395
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:45395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:43.723991 9986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:43.726029 9986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:43.738327 9999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:43.738353 10002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:43.738391 10000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:43.739275 9986 server_base.cc:1034] running on GCE node
I20260811 00:17:43.740154 9986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:43.741631 9986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:43.742892 9986 hybrid_clock.cc:648] HybridClock initialized: now 1786407463742821 us; error 41 us; skew 500 ppm
I20260811 00:17:43.743257 9986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:17:43.752467 9986 webserver.cc:460] Webserver started at http://127.7.94.62:38155/ using document root <none> and password file <none>
I20260811 00:17:43.753285 9986 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:43.753424 9986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:43.753777 9986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:43.756889 9986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "6860168532c244ff99f08c09d8a5412a"
format_stamp: "Formatted at 2026-08-11 00:17:43 on dist-test-slave-8hhm"
server_key: "cbbaa8fd7f04f2fb8b9531e5f3d618ef"
server_key_iv: "7e31a0cde2577e04199f506637d536da"
server_key_version: "encryptionkey@0"
I20260811 00:17:43.757746 9986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "6860168532c244ff99f08c09d8a5412a"
format_stamp: "Formatted at 2026-08-11 00:17:43 on dist-test-slave-8hhm"
server_key: "cbbaa8fd7f04f2fb8b9531e5f3d618ef"
server_key_iv: "7e31a0cde2577e04199f506637d536da"
server_key_version: "encryptionkey@0"
I20260811 00:17:43.762996 9986 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:17:43.766508 10011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:43.767864 9986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:17:43.768054 9986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "6860168532c244ff99f08c09d8a5412a"
format_stamp: "Formatted at 2026-08-11 00:17:43 on dist-test-slave-8hhm"
server_key: "cbbaa8fd7f04f2fb8b9531e5f3d618ef"
server_key_iv: "7e31a0cde2577e04199f506637d536da"
server_key_version: "encryptionkey@0"
I20260811 00:17:43.768285 9986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:43.788398 9986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:43.789256 9986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:43.789551 9986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:43.808362 9986 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:45395
I20260811 00:17:43.808390 10092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:45395 every 8 connection(s)
I20260811 00:17:43.809738 9986 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:17:43.813544 10093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:43.816488 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 9986
I20260811 00:17:43.816674 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:17:43.817242 7544 external_mini_cluster.cc:1428] Setting key e19082d7552ed8d1a1bf1bcfd9fc32c5
I20260811 00:17:43.825695 10093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a: Bootstrap starting.
I20260811 00:17:43.828908 10093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:43.830092 10093 log.cc:824] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:43.831023 10097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:43.819238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43006 (local address 127.7.94.62:45395)
0811 00:17:43.820280 (+ 1042us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:43.820291 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:43.820333 (+ 42us) server_negotiation.cc:407] Connection header received
0811 00:17:43.822122 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:43.822146 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:43.822605 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:43.823005 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:43.823604 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:43.824850 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:43.826342 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:43.827032 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:43.827262 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:43.828416 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:43.828424 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:43.828436 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:43.828992 (+ 556us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:43.829129 (+ 137us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:43.829157 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:43.829348 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:43.829552 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 00:17:43.829844 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":383,"threads_started":1}
I20260811 00:17:43.833202 10093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a: No bootstrap required, opened a new log
I20260811 00:17:43.840972 10093 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6860168532c244ff99f08c09d8a5412a" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45395 } }
I20260811 00:17:43.841449 10093 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:43.841562 10093 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6860168532c244ff99f08c09d8a5412a, State: Initialized, Role: FOLLOWER
I20260811 00:17:43.842125 10093 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6860168532c244ff99f08c09d8a5412a" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45395 } }
I20260811 00:17:43.842365 10093 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:17:43.842521 10093 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:17:43.842684 10093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:43.844830 10093 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6860168532c244ff99f08c09d8a5412a" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45395 } }
I20260811 00:17:43.845439 10093 leader_election.cc:302] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6860168532c244ff99f08c09d8a5412a; no voters:
I20260811 00:17:43.846020 10093 leader_election.cc:288] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:17:43.846181 10100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:17:43.847545 10100 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a [term 1 LEADER]: Becoming Leader. State: Replica: 6860168532c244ff99f08c09d8a5412a, State: Running, Role: LEADER
I20260811 00:17:43.848131 10093 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:17:43.848019 10100 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6860168532c244ff99f08c09d8a5412a" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45395 } }
I20260811 00:17:43.850998 10104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6860168532c244ff99f08c09d8a5412a. Latest consensus state: current_term: 1 leader_uuid: "6860168532c244ff99f08c09d8a5412a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6860168532c244ff99f08c09d8a5412a" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45395 } } }
I20260811 00:17:43.851507 10104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:43.851055 10101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6860168532c244ff99f08c09d8a5412a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6860168532c244ff99f08c09d8a5412a" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 45395 } } }
I20260811 00:17:43.851732 10101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:43.854081 10117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:17:43.858770 10117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:17:43.867085 10117 catalog_manager.cc:1347] Generated new cluster ID: 1ef52f23bbd2498baa521bcfc793648d
I20260811 00:17:43.867197 10117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:17:43.903002 10117 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:17:43.904950 10117 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:17:43.916703 10117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a: Generated new TSK 0
I20260811 00:17:43.917805 10117 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:17:43.926077 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.1:0
--local_ip_for_outbound_sockets=127.7.94.1
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45395
--builtin_ntp_servers=127.7.94.20:42115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 00:17:44.301534 10130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:44.301823 10130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:44.301980 10130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:44.309044 10130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:44.309141 10130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:44.309257 10130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.1
I20260811 00:17:44.318236 10130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:42115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.94.1
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.1
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:44.319540 10130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:44.321501 10130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:44.334744 10137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:44.334789 10141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:44.334827 10138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:44.335834 10130 server_base.cc:1034] running on GCE node
I20260811 00:17:44.336427 10130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:44.337428 10130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:44.338670 10130 hybrid_clock.cc:648] HybridClock initialized: now 1786407464338632 us; error 48 us; skew 500 ppm
I20260811 00:17:44.339089 10130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:17:44.341396 10130 webserver.cc:460] Webserver started at http://127.7.94.1:33677/ using document root <none> and password file <none>
I20260811 00:17:44.342087 10130 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:44.342219 10130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:44.342549 10130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:44.345504 10130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-0/data/instance:
uuid: "c2ef9800da9f4e4a937dee0a1cd7745b"
format_stamp: "Formatted at 2026-08-11 00:17:44 on dist-test-slave-8hhm"
server_key: "84a0c93f51c090a981ed73efbd0af801"
server_key_iv: "4b7a8341f62aa16c52e0386bb2b12731"
server_key_version: "encryptionkey@0"
I20260811 00:17:44.346311 10130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance:
uuid: "c2ef9800da9f4e4a937dee0a1cd7745b"
format_stamp: "Formatted at 2026-08-11 00:17:44 on dist-test-slave-8hhm"
server_key: "84a0c93f51c090a981ed73efbd0af801"
server_key_iv: "4b7a8341f62aa16c52e0386bb2b12731"
server_key_version: "encryptionkey@0"
I20260811 00:17:44.351773 10130 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:17:44.355656 10156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:44.356974 10130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:17:44.357156 10130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-0/wal
uuid: "c2ef9800da9f4e4a937dee0a1cd7745b"
format_stamp: "Formatted at 2026-08-11 00:17:44 on dist-test-slave-8hhm"
server_key: "84a0c93f51c090a981ed73efbd0af801"
server_key_iv: "4b7a8341f62aa16c52e0386bb2b12731"
server_key_version: "encryptionkey@0"
I20260811 00:17:44.357472 10130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:44.383462 10130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:44.384526 10130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:44.384778 10130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:44.385819 10130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:44.387282 10130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:17:44.387368 10130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:44.387449 10130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:44.387504 10130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:44.423477 10130 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.1:33363
I20260811 00:17:44.423564 10345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.1:33363 every 8 connection(s)
I20260811 00:17:44.424831 10130 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-0/data/info.pb
I20260811 00:17:44.435052 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 10130
I20260811 00:17:44.435226 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-0/wal/instance
I20260811 00:17:44.435698 7544 external_mini_cluster.cc:1428] Setting key ae8ae3157beaba83abc759c59720d22b
I20260811 00:17:44.437747 10350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:44.427429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:42431 (local address 127.7.94.62:45395)
0811 00:17:44.427921 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:44.427926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:44.429273 (+ 1347us) server_negotiation.cc:407] Connection header received
0811 00:17:44.431100 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:44.431106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:44.431195 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:44.431336 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:44.432427 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:44.433141 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:44.434764 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:44.435247 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:44.435389 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:44.436830 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:44.436835 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:44.436838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:44.436855 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:44.436895 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:44.436907 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:44.437005 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:44.437271 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 00:17:44.437367 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":271,"threads_started":1}
I20260811 00:17:44.438495 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.2:0
--local_ip_for_outbound_sockets=127.7.94.2
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45395
--builtin_ntp_servers=127.7.94.20:42115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 00:17:44.438866 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:44.427914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:45395 (local address 127.7.94.1:42431)
0811 00:17:44.429001 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:44.429080 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:44.430837 (+ 1757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:44.431578 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:44.431587 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:44.431675 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:17:44.432253 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:44.432276 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:44.433369 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:44.433374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:44.434607 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:44.434620 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:44.435490 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:44.435497 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:44.435734 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:44.436485 (+ 751us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:17:44.436488 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:17:44.436539 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:17:44.436617 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:17:44.437007 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:17:44.437022 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:17:44.437031 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 00:17:44.437244 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 00:17:44.437656 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":289,"threads_started":1}
I20260811 00:17:44.440421 10347 heartbeater.cc:347] Connected to a master server at 127.7.94.62:45395
I20260811 00:17:44.440770 10347 heartbeater.cc:464] Registering TS with master...
I20260811 00:17:44.441721 10347 heartbeater.cc:511] Master 127.7.94.62:45395 requested a full tablet report, sending...
I20260811 00:17:44.443871 10033 ts_manager.cc:194] Registered new tserver with Master: c2ef9800da9f4e4a937dee0a1cd7745b (127.7.94.1:33363)
I20260811 00:17:44.445446 10033 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.94.1:42431
W20260811 00:17:44.784029 10352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:44.784305 10352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:44.784446 10352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:44.791532 10352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:44.791630 10352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:44.791790 10352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.2
I20260811 00:17:44.800515 10352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:42115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.94.2
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.2
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:44.801968 10352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:44.803779 10352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:44.817690 10360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:44.817662 10361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:44.817968 10363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:44.818356 10352 server_base.cc:1034] running on GCE node
I20260811 00:17:44.819087 10352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:44.820142 10352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:44.821316 10352 hybrid_clock.cc:648] HybridClock initialized: now 1786407464821253 us; error 49 us; skew 500 ppm
I20260811 00:17:44.821648 10352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:17:44.824086 10352 webserver.cc:460] Webserver started at http://127.7.94.2:38775/ using document root <none> and password file <none>
I20260811 00:17:44.824792 10352 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:44.824906 10352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:44.825300 10352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:44.828205 10352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-1/data/instance:
uuid: "c2418259ce0e4fe5863ff85ad593a1c2"
format_stamp: "Formatted at 2026-08-11 00:17:44 on dist-test-slave-8hhm"
server_key: "ff835bf4975b878aced2daccc8a5c3e5"
server_key_iv: "a843b630d459290ef8c4415a9281af74"
server_key_version: "encryptionkey@0"
I20260811 00:17:44.829025 10352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance:
uuid: "c2418259ce0e4fe5863ff85ad593a1c2"
format_stamp: "Formatted at 2026-08-11 00:17:44 on dist-test-slave-8hhm"
server_key: "ff835bf4975b878aced2daccc8a5c3e5"
server_key_iv: "a843b630d459290ef8c4415a9281af74"
server_key_version: "encryptionkey@0"
I20260811 00:17:44.834015 10352 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 00:17:44.837437 10380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:44.838631 10352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:17:44.838797 10352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-1/wal
uuid: "c2418259ce0e4fe5863ff85ad593a1c2"
format_stamp: "Formatted at 2026-08-11 00:17:44 on dist-test-slave-8hhm"
server_key: "ff835bf4975b878aced2daccc8a5c3e5"
server_key_iv: "a843b630d459290ef8c4415a9281af74"
server_key_version: "encryptionkey@0"
I20260811 00:17:44.839004 10352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:44.874161 10352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:44.875072 10352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:44.875362 10352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:44.876364 10352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:44.878024 10352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:17:44.878146 10352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:44.878285 10352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:44.878371 10352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:44.916306 10352 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.2:43191
I20260811 00:17:44.916395 10561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.2:43191 every 8 connection(s)
I20260811 00:17:44.917956 10352 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-1/data/info.pb
I20260811 00:17:44.924288 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 10352
I20260811 00:17:44.924469 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-1/wal/instance
I20260811 00:17:44.925086 7544 external_mini_cluster.cc:1428] Setting key d5a971debd71ada0e4f8f0e6e28fe9cf
I20260811 00:17:44.928092 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.94.3:0
--local_ip_for_outbound_sockets=127.7.94.3
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45395
--builtin_ntp_servers=127.7.94.20:42115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 00:17:44.931691 10350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:44.920945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.2:50775 (local address 127.7.94.62:45395)
0811 00:17:44.921169 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:44.921174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:44.922732 (+ 1558us) server_negotiation.cc:407] Connection header received
0811 00:17:44.924892 (+ 2160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:44.924897 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:44.924967 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:44.925058 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:44.926344 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:44.927178 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:44.928864 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:44.929336 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:44.929469 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:44.930867 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:44.930872 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:44.930875 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:44.930891 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:44.930922 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:44.930934 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:44.931028 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:44.931271 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 00:17:44.931363 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 00:17:44.932703 10564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:44.921637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:45395 (local address 127.7.94.2:50775)
0811 00:17:44.922575 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:44.922617 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:44.924568 (+ 1951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:44.925363 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:44.925374 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:44.925490 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:17:44.926150 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:44.926194 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:44.927343 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:44.927348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:44.928712 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:44.928724 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:44.929570 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:44.929579 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:44.929804 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:44.930527 (+ 723us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:17:44.930530 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:17:44.930570 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:17:44.930651 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:17:44.931029 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:17:44.931071 (+ 42us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:17:44.931078 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 00:17:44.931259 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 00:17:44.931644 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":266,"threads_started":1}
I20260811 00:17:44.934238 10562 heartbeater.cc:347] Connected to a master server at 127.7.94.62:45395
I20260811 00:17:44.934624 10562 heartbeater.cc:464] Registering TS with master...
I20260811 00:17:44.935436 10562 heartbeater.cc:511] Master 127.7.94.62:45395 requested a full tablet report, sending...
I20260811 00:17:44.937002 10033 ts_manager.cc:194] Registered new tserver with Master: c2418259ce0e4fe5863ff85ad593a1c2 (127.7.94.2:43191)
I20260811 00:17:44.937862 10033 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.94.2:50775
W20260811 00:17:45.288784 10565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:45.289093 10565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:45.289188 10565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:45.296460 10565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:45.296567 10565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:45.296681 10565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.94.3
I20260811 00:17:45.305846 10565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:42115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.94.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.94.3
--webserver_port=0
--tserver_master_addrs=127.7.94.62:45395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.94.3
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:45.307128 10565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:45.308960 10565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:45.322819 10570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:45.322894 10573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:45.322850 10571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:45.323994 10565 server_base.cc:1034] running on GCE node
I20260811 00:17:45.324640 10565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:45.325739 10565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:45.327029 10565 hybrid_clock.cc:648] HybridClock initialized: now 1786407465326967 us; error 43 us; skew 500 ppm
I20260811 00:17:45.327390 10565 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 00:17:45.329918 10565 webserver.cc:460] Webserver started at http://127.7.94.3:42329/ using document root <none> and password file <none>
I20260811 00:17:45.330639 10565 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:45.330783 10565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:45.331137 10565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:45.334203 10565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-2/data/instance:
uuid: "a21afbf8ee3e4b8991237a5bfdfcfcda"
format_stamp: "Formatted at 2026-08-11 00:17:45 on dist-test-slave-8hhm"
server_key: "da07d96feb1c91a10ff97fb29842f165"
server_key_iv: "b3c5c6d177c9171007e92c78549309ae"
server_key_version: "encryptionkey@0"
I20260811 00:17:45.335059 10565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance:
uuid: "a21afbf8ee3e4b8991237a5bfdfcfcda"
format_stamp: "Formatted at 2026-08-11 00:17:45 on dist-test-slave-8hhm"
server_key: "da07d96feb1c91a10ff97fb29842f165"
server_key_iv: "b3c5c6d177c9171007e92c78549309ae"
server_key_version: "encryptionkey@0"
I20260811 00:17:45.340538 10565 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:17:45.344146 10585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:45.345468 10565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 00:17:45.345638 10565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-2/wal
uuid: "a21afbf8ee3e4b8991237a5bfdfcfcda"
format_stamp: "Formatted at 2026-08-11 00:17:45 on dist-test-slave-8hhm"
server_key: "da07d96feb1c91a10ff97fb29842f165"
server_key_iv: "b3c5c6d177c9171007e92c78549309ae"
server_key_version: "encryptionkey@0"
I20260811 00:17:45.345853 10565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:45.375388 10565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:45.376250 10565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:45.376528 10565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:45.377497 10565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 00:17:45.378955 10565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 00:17:45.379070 10565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:45.379196 10565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 00:17:45.379280 10565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:45.412271 10565 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.3:43705
I20260811 00:17:45.412431 10765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.3:43705 every 8 connection(s)
I20260811 00:17:45.413676 10565 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-2/data/info.pb
I20260811 00:17:45.423159 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 10565
I20260811 00:17:45.423368 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-2/wal/instance
I20260811 00:17:45.423924 7544 external_mini_cluster.cc:1428] Setting key f02df345c136bb8b25d35598b268db4f
I20260811 00:17:45.427464 10350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:45.416280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.3:55139 (local address 127.7.94.62:45395)
0811 00:17:45.416461 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:45.416466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:45.417894 (+ 1428us) server_negotiation.cc:407] Connection header received
0811 00:17:45.419622 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:45.419628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:45.419730 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:45.419839 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:45.420957 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.421830 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.423828 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.424487 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.424642 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:45.426289 (+ 1647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:45.426293 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:45.426297 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:45.426334 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:45.426364 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:45.426377 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:45.426473 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:45.427022 (+ 549us) server_negotiation.cc:299] Negotiation successful
0811 00:17:45.427120 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:17:45.428202 10771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:45.416782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:45395 (local address 127.7.94.3:55139)
0811 00:17:45.417741 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:45.417778 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:45.419361 (+ 1583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:45.420088 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:45.420096 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:45.420178 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:17:45.420788 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:45.420811 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.421982 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.421994 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:45.423664 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:45.423679 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.424791 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.424801 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:45.425087 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:45.425957 (+ 870us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 00:17:45.425961 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:17:45.426021 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 00:17:45.426091 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:17:45.426475 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:17:45.426488 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:17:45.426498 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 00:17:45.426699 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 00:17:45.427101 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":326,"threads_started":1}
I20260811 00:17:45.429616 10766 heartbeater.cc:347] Connected to a master server at 127.7.94.62:45395
I20260811 00:17:45.429909 10766 heartbeater.cc:464] Registering TS with master...
I20260811 00:17:45.430619 10766 heartbeater.cc:511] Master 127.7.94.62:45395 requested a full tablet report, sending...
I20260811 00:17:45.431959 10033 ts_manager.cc:194] Registered new tserver with Master: a21afbf8ee3e4b8991237a5bfdfcfcda (127.7.94.3:43705)
I20260811 00:17:45.432758 10033 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.94.3:55139
I20260811 00:17:45.439034 7544 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 00:17:45.448532 10350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:45.443019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43012 (local address 127.7.94.62:45395)
0811 00:17:45.443263 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:45.443273 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:45.443754 (+ 481us) server_negotiation.cc:407] Connection header received
0811 00:17:45.443879 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:45.443882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:45.443939 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:45.444015 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:45.444513 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.445157 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.446440 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.446824 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.446910 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:45.447833 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:45.447837 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:45.447840 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:45.447855 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:45.447884 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:45.447894 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:45.447959 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:45.448127 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 00:17:45.448240 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 00:17:45.449021 10347 heartbeater.cc:503] Master 127.7.94.62:45395 was elected leader, sending a full tablet report...
I20260811 00:17:45.452489 10033 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43012:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 00:17:45.459002 10033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 00:17:45.480741 10800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:45.471239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41106 (local address 127.7.94.2:43191)
0811 00:17:45.472217 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:45.472225 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:45.472256 (+ 31us) server_negotiation.cc:407] Connection header received
0811 00:17:45.472361 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:45.472368 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:45.472604 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:45.473435 (+ 831us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:45.473697 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.475751 (+ 2054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.478123 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.479385 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.479521 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:45.479793 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:45.480254 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 00:17:45.480420 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":794,"thread_start_us":983,"threads_started":1}
I20260811 00:17:45.480777 10797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:45.470559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:43191 (local address 127.0.0.1:41106)
0811 00:17:45.471959 (+ 1400us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:45.471977 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:45.472104 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:45.472920 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:45.472930 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:45.472971 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:45.473463 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:45.473477 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.475903 (+ 2426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.475908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:45.477940 (+ 2032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:45.477952 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.479892 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.479897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:45.480023 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:45.480053 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 00:17:45.480200 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 00:17:45.480374 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1267,"mutex_wait_us":215,"thread_start_us":1074,"threads_started":1}
I20260811 00:17:45.481389 10802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:45.472028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:43705 (local address 127.0.0.1:37946)
0811 00:17:45.472794 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:45.472810 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:45.472923 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:45.474022 (+ 1099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:45.474026 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:45.474050 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:45.474390 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:45.474400 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.476069 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.476074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:45.478568 (+ 2494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:45.478583 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.480610 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.480617 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:45.480800 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:45.480838 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 00:17:45.480902 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 00:17:45.481016 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":189,"thread_start_us":687,"threads_started":1}
I20260811 00:17:45.481887 10789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:45.470291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:33363 (local address 127.0.0.1:52516)
0811 00:17:45.471572 (+ 1281us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:45.471624 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:45.471924 (+ 300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:45.472980 (+ 1056us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:45.472986 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:45.473011 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:45.473567 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:45.473582 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.475476 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.475481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:45.478573 (+ 3092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:45.478583 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.481135 (+ 2552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.481143 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:45.481354 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:45.481399 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 00:17:45.481491 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 00:17:45.481610 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1027,"mutex_wait_us":151,"thread_start_us":871,"threads_started":1}
I20260811 00:17:45.482118 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:45.472556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37946 (local address 127.7.94.3:43705)
0811 00:17:45.473496 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:45.473513 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:45.473547 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:17:45.473657 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:45.473667 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:45.473848 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:45.474029 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:45.474580 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.475904 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.478826 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.480384 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.481345 (+ 961us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:45.481590 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:45.481653 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 00:17:45.481800 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":769,"thread_start_us":595,"threads_started":1}
I20260811 00:17:45.482686 10794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:45.470487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52516 (local address 127.7.94.1:33363)
0811 00:17:45.471165 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:45.471176 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:45.471758 (+ 582us) server_negotiation.cc:407] Connection header received
0811 00:17:45.472215 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:45.472226 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:45.472525 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:45.472696 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:45.473745 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.475339 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.478826 (+ 3487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.480384 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.480524 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:45.480787 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:45.482131 (+ 1344us) server_negotiation.cc:299] Negotiation successful
0811 00:17:45.482325 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":333,"threads_started":1}
I20260811 00:17:45.485433 10661 tablet_service.cc:1467] Processing CreateTablet for tablet d94e660ffee14d15ae22a11c6f43f049 (DEFAULT_TABLE table=test-table [id=876a21a942fd4006bcc28e8c9d0dc15b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:17:45.485347 10453 tablet_service.cc:1467] Processing CreateTablet for tablet d94e660ffee14d15ae22a11c6f43f049 (DEFAULT_TABLE table=test-table [id=876a21a942fd4006bcc28e8c9d0dc15b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:17:45.486560 10661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d94e660ffee14d15ae22a11c6f43f049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:45.486189 10239 tablet_service.cc:1467] Processing CreateTablet for tablet d94e660ffee14d15ae22a11c6f43f049 (DEFAULT_TABLE table=test-table [id=876a21a942fd4006bcc28e8c9d0dc15b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 00:17:45.486733 10453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d94e660ffee14d15ae22a11c6f43f049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:45.487409 10239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d94e660ffee14d15ae22a11c6f43f049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:45.497452 10805 tablet_bootstrap.cc:492] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda: Bootstrap starting.
I20260811 00:17:45.500804 10807 tablet_bootstrap.cc:492] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2: Bootstrap starting.
I20260811 00:17:45.501572 10805 tablet_bootstrap.cc:654] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:45.502799 10809 tablet_bootstrap.cc:492] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b: Bootstrap starting.
I20260811 00:17:45.502985 10805 log.cc:824] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:45.504328 10807 tablet_bootstrap.cc:654] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:45.505667 10807 log.cc:824] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:45.505970 10809 tablet_bootstrap.cc:654] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:45.506475 10805 tablet_bootstrap.cc:492] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda: No bootstrap required, opened a new log
I20260811 00:17:45.506738 10805 ts_tablet_manager.cc:1397] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.001s
I20260811 00:17:45.507148 10809 log.cc:824] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:45.508638 10807 tablet_bootstrap.cc:492] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2: No bootstrap required, opened a new log
I20260811 00:17:45.508878 10807 ts_tablet_manager.cc:1397] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260811 00:17:45.509620 10809 tablet_bootstrap.cc:492] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b: No bootstrap required, opened a new log
I20260811 00:17:45.509862 10809 ts_tablet_manager.cc:1397] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 00:17:45.516732 10809 raft_consensus.cc:348] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2ef9800da9f4e4a937dee0a1cd7745b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33363 } } peers { permanent_uuid: "c2418259ce0e4fe5863ff85ad593a1c2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43191 } } peers { permanent_uuid: "a21afbf8ee3e4b8991237a5bfdfcfcda" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43705 } }
I20260811 00:17:45.517083 10809 raft_consensus.cc:374] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:45.517185 10809 raft_consensus.cc:729] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2ef9800da9f4e4a937dee0a1cd7745b, State: Initialized, Role: FOLLOWER
I20260811 00:17:45.517802 10809 consensus_queue.cc:260] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2ef9800da9f4e4a937dee0a1cd7745b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33363 } } peers { permanent_uuid: "c2418259ce0e4fe5863ff85ad593a1c2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43191 } } peers { permanent_uuid: "a21afbf8ee3e4b8991237a5bfdfcfcda" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43705 } }
I20260811 00:17:45.517656 10805 raft_consensus.cc:348] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2ef9800da9f4e4a937dee0a1cd7745b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33363 } } peers { permanent_uuid: "c2418259ce0e4fe5863ff85ad593a1c2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43191 } } peers { permanent_uuid: "a21afbf8ee3e4b8991237a5bfdfcfcda" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43705 } }
I20260811 00:17:45.518062 10805 raft_consensus.cc:374] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:45.518162 10805 raft_consensus.cc:729] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a21afbf8ee3e4b8991237a5bfdfcfcda, State: Initialized, Role: FOLLOWER
I20260811 00:17:45.518749 10805 consensus_queue.cc:260] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2ef9800da9f4e4a937dee0a1cd7745b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33363 } } peers { permanent_uuid: "c2418259ce0e4fe5863ff85ad593a1c2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43191 } } peers { permanent_uuid: "a21afbf8ee3e4b8991237a5bfdfcfcda" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43705 } }
I20260811 00:17:45.519642 10809 ts_tablet_manager.cc:1428] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260811 00:17:45.520638 10766 heartbeater.cc:503] Master 127.7.94.62:45395 was elected leader, sending a full tablet report...
I20260811 00:17:45.521720 10805 ts_tablet_manager.cc:1428] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260811 00:17:45.521966 10807 raft_consensus.cc:348] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2ef9800da9f4e4a937dee0a1cd7745b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33363 } } peers { permanent_uuid: "c2418259ce0e4fe5863ff85ad593a1c2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43191 } } peers { permanent_uuid: "a21afbf8ee3e4b8991237a5bfdfcfcda" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43705 } }
I20260811 00:17:45.522421 10807 raft_consensus.cc:374] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:45.522522 10807 raft_consensus.cc:729] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2418259ce0e4fe5863ff85ad593a1c2, State: Initialized, Role: FOLLOWER
I20260811 00:17:45.523170 10807 consensus_queue.cc:260] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2ef9800da9f4e4a937dee0a1cd7745b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33363 } } peers { permanent_uuid: "c2418259ce0e4fe5863ff85ad593a1c2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43191 } } peers { permanent_uuid: "a21afbf8ee3e4b8991237a5bfdfcfcda" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43705 } }
I20260811 00:17:45.524439 10562 heartbeater.cc:503] Master 127.7.94.62:45395 was elected leader, sending a full tablet report...
I20260811 00:17:45.524967 10807 ts_tablet_manager.cc:1428] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2: Time spent starting tablet: real 0.016s user 0.012s sys 0.002s
W20260811 00:17:45.665519 10770 tablet.cc:2369] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:17:45.670840 10563 tablet.cc:2369] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 00:17:45.677259 10348 tablet.cc:2369] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:17:45.692634 10813 raft_consensus.cc:482] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 00:17:45.692896 10813 raft_consensus.cc:504] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2ef9800da9f4e4a937dee0a1cd7745b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33363 } } peers { permanent_uuid: "c2418259ce0e4fe5863ff85ad593a1c2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43191 } } peers { permanent_uuid: "a21afbf8ee3e4b8991237a5bfdfcfcda" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43705 } }
I20260811 00:17:45.694643 10813 leader_election.cc:288] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2418259ce0e4fe5863ff85ad593a1c2 (127.7.94.2:43191), a21afbf8ee3e4b8991237a5bfdfcfcda (127.7.94.3:43705)
I20260811 00:17:45.701140 10800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:45.695103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:51869 (local address 127.7.94.2:43191)
0811 00:17:45.695291 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:45.695296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:45.695889 (+ 593us) server_negotiation.cc:407] Connection header received
0811 00:17:45.696140 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:45.696149 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:45.696258 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:45.696321 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:45.697024 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.697776 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.699528 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.700246 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.700387 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:45.700477 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:45.700748 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 00:17:45.700869 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 00:17:45.701275 10817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:45.695012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.2:43191 (local address 127.7.94.1:51869)
0811 00:17:45.695769 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:45.695790 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:45.695939 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:45.696529 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:45.696532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:45.696557 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:45.696885 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:45.696892 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.697903 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.697911 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:45.699337 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:45.699348 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.700424 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.700430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:45.700579 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:45.700602 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 00:17:45.700710 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 00:17:45.700832 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"mutex_wait_us":43,"thread_start_us":427,"threads_started":1}
I20260811 00:17:45.701653 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:45.696302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.94.1:35531 (local address 127.7.94.3:43705)
0811 00:17:45.696521 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:45.696528 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:45.696696 (+ 168us) server_negotiation.cc:407] Connection header received
0811 00:17:45.696931 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:45.696937 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:45.697002 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:45.697110 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 00:17:45.697645 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.698320 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.699981 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.700731 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.700871 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:45.700972 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:45.701279 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 00:17:45.701403 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 00:17:45.701736 10819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:45.695918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.3:43705 (local address 127.7.94.1:35531)
0811 00:17:45.696556 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:45.696573 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:17:45.696777 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:17:45.697120 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:17:45.697124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:17:45.697146 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 00:17:45.697484 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:45.697492 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.698470 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.698475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:45.699810 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:17:45.699822 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.700936 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.700943 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:17:45.701104 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:45.701126 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 00:17:45.701253 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 00:17:45.701381 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":1,"thread_start_us":304,"threads_started":1}
I20260811 00:17:45.702291 10484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d94e660ffee14d15ae22a11c6f43f049" candidate_uuid: "c2ef9800da9f4e4a937dee0a1cd7745b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2418259ce0e4fe5863ff85ad593a1c2" is_pre_election: true
I20260811 00:17:45.702845 10484 raft_consensus.cc:2454] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2ef9800da9f4e4a937dee0a1cd7745b in term 0.
I20260811 00:17:45.703598 10158 leader_election.cc:302] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2418259ce0e4fe5863ff85ad593a1c2, c2ef9800da9f4e4a937dee0a1cd7745b; no voters:
I20260811 00:17:45.703531 10702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d94e660ffee14d15ae22a11c6f43f049" candidate_uuid: "c2ef9800da9f4e4a937dee0a1cd7745b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a21afbf8ee3e4b8991237a5bfdfcfcda" is_pre_election: true
I20260811 00:17:45.703985 10813 raft_consensus.cc:2781] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 00:17:45.704052 10702 raft_consensus.cc:2454] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2ef9800da9f4e4a937dee0a1cd7745b in term 0.
I20260811 00:17:45.704167 10813 raft_consensus.cc:482] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 00:17:45.704260 10813 raft_consensus.cc:3037] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:45.706522 10813 raft_consensus.cc:504] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2ef9800da9f4e4a937dee0a1cd7745b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33363 } } peers { permanent_uuid: "c2418259ce0e4fe5863ff85ad593a1c2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43191 } } peers { permanent_uuid: "a21afbf8ee3e4b8991237a5bfdfcfcda" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43705 } }
I20260811 00:17:45.707139 10813 leader_election.cc:288] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [CANDIDATE]: Term 1 election: Requested vote from peers c2418259ce0e4fe5863ff85ad593a1c2 (127.7.94.2:43191), a21afbf8ee3e4b8991237a5bfdfcfcda (127.7.94.3:43705)
I20260811 00:17:45.707638 10484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d94e660ffee14d15ae22a11c6f43f049" candidate_uuid: "c2ef9800da9f4e4a937dee0a1cd7745b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2418259ce0e4fe5863ff85ad593a1c2"
I20260811 00:17:45.707846 10484 raft_consensus.cc:3037] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:45.707980 10702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d94e660ffee14d15ae22a11c6f43f049" candidate_uuid: "c2ef9800da9f4e4a937dee0a1cd7745b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a21afbf8ee3e4b8991237a5bfdfcfcda"
I20260811 00:17:45.708185 10702 raft_consensus.cc:3037] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:45.710214 10484 raft_consensus.cc:2454] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2ef9800da9f4e4a937dee0a1cd7745b in term 1.
I20260811 00:17:45.710561 10158 leader_election.cc:302] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2418259ce0e4fe5863ff85ad593a1c2, c2ef9800da9f4e4a937dee0a1cd7745b; no voters:
I20260811 00:17:45.710518 10702 raft_consensus.cc:2454] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2ef9800da9f4e4a937dee0a1cd7745b in term 1.
I20260811 00:17:45.710848 10813 raft_consensus.cc:2781] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:17:45.711294 10813 raft_consensus.cc:686] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [term 1 LEADER]: Becoming Leader. State: Replica: c2ef9800da9f4e4a937dee0a1cd7745b, State: Running, Role: LEADER
I20260811 00:17:45.711799 10813 consensus_queue.cc:237] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2ef9800da9f4e4a937dee0a1cd7745b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33363 } } peers { permanent_uuid: "c2418259ce0e4fe5863ff85ad593a1c2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43191 } } peers { permanent_uuid: "a21afbf8ee3e4b8991237a5bfdfcfcda" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43705 } }
I20260811 00:17:45.715997 10033 catalog_manager.cc:5495] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b reported cstate change: term changed from 0 to 1, leader changed from <none> to c2ef9800da9f4e4a937dee0a1cd7745b (127.7.94.1). New cstate: current_term: 1 leader_uuid: "c2ef9800da9f4e4a937dee0a1cd7745b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2ef9800da9f4e4a937dee0a1cd7745b" member_type: VOTER last_known_addr { host: "127.7.94.1" port: 33363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2418259ce0e4fe5863ff85ad593a1c2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a21afbf8ee3e4b8991237a5bfdfcfcda" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43705 } health_report { overall_health: UNKNOWN } } }
I20260811 00:17:45.775735 10794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:45.770413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52524 (local address 127.7.94.1:33363)
0811 00:17:45.770647 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:45.770651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:45.770685 (+ 34us) server_negotiation.cc:407] Connection header received
0811 00:17:45.770894 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:45.770898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:45.770990 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:45.771123 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:17:45.771432 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.772211 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.773731 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.774145 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.774237 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:45.774657 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:17:45.775186 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:17:45.775296 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:45.775383 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 00:17:45.775444 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 00:17:45.783864 10702 raft_consensus.cc:1261] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda [term 1 FOLLOWER]: Refusing update from remote peer c2ef9800da9f4e4a937dee0a1cd7745b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:17:45.783905 10484 raft_consensus.cc:1261] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2 [term 1 FOLLOWER]: Refusing update from remote peer c2ef9800da9f4e4a937dee0a1cd7745b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:17:45.784858 10820 consensus_queue.cc:1035] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2418259ce0e4fe5863ff85ad593a1c2" member_type: VOTER last_known_addr { host: "127.7.94.2" port: 43191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:17:45.785288 10813 consensus_queue.cc:1035] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a21afbf8ee3e4b8991237a5bfdfcfcda" member_type: VOTER last_known_addr { host: "127.7.94.3" port: 43705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:17:45.795722 10829 mvcc.cc:205] Tried to move back new op lower bound from 7317124979841531904 to 7317124979558981632. Current Snapshot: MvccSnapshot[applied={T|T < 7317124979841531904}]
I20260811 00:17:45.796458 10834 mvcc.cc:205] Tried to move back new op lower bound from 7317124979841531904 to 7317124979558981632. Current Snapshot: MvccSnapshot[applied={T|T < 7317124979841531904}]
I20260811 00:17:45.811630 10033 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:17:45.811930 10033 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 00:17:45.815733 10033 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6860168532c244ff99f08c09d8a5412a: Sending DeleteTablet for 3 replicas of tablet d94e660ffee14d15ae22a11c6f43f049
I20260811 00:17:45.816762 10239 tablet_service.cc:1514] Processing DeleteTablet for tablet d94e660ffee14d15ae22a11c6f43f049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:17:45 UTC) from {username='slave'} at 127.0.0.1:52516
I20260811 00:17:45.816854 10453 tablet_service.cc:1514] Processing DeleteTablet for tablet d94e660ffee14d15ae22a11c6f43f049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:17:45 UTC) from {username='slave'} at 127.0.0.1:41106
I20260811 00:17:45.817056 10661 tablet_service.cc:1514] Processing DeleteTablet for tablet d94e660ffee14d15ae22a11c6f43f049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 00:17:45 UTC) from {username='slave'} at 127.0.0.1:37946
I20260811 00:17:45.818877 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 10130
I20260811 00:17:45.835048 10848 tablet_replica.cc:330] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda: stopping tablet replica
I20260811 00:17:45.835608 10848 raft_consensus.cc:2229] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:17:45.836217 10848 raft_consensus.cc:2258] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:17:45.837320 10846 tablet_replica.cc:330] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b: stopping tablet replica
I20260811 00:17:45.837721 10846 raft_consensus.cc:2229] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [term 1 LEADER]: Raft consensus shutting down.
I20260811 00:17:45.838435 10846 raft_consensus.cc:2258] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:17:45.840574 10848 ts_tablet_manager.cc:1905] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:17:45.842353 10794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:45.820241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52526 (local address 127.7.94.1:33363)
0811 00:17:45.820435 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:45.820442 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:45.828140 (+ 7698us) server_negotiation.cc:407] Connection header received
0811 00:17:45.828362 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:45.828368 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:45.828460 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:45.828562 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:45.833931 (+ 5369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.835159 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.837205 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.837949 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.838077 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:45.840691 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:45.840702 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:45.840718 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:45.841470 (+ 752us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:45.841532 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:45.841558 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:45.841676 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:45.841872 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 00:17:45.841966 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 00:17:45.843082 10846 ts_tablet_manager.cc:1905] T d94e660ffee14d15ae22a11c6f43f049 P c2ef9800da9f4e4a937dee0a1cd7745b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:17:45.843355 10341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:45.848843 10847 tablet_replica.cc:330] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2: stopping tablet replica
I20260811 00:17:45.849293 10847 raft_consensus.cc:2229] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 00:17:45.849833 10847 raft_consensus.cc:2258] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 00:17:45.851964 10847 ts_tablet_manager.cc:1905] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 00:17:45.853936 10848 ts_tablet_manager.cc:1918] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:17:45.854142 10848 log.cc:1196] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-2/wal/wals/d94e660ffee14d15ae22a11c6f43f049
I20260811 00:17:45.854753 10848 ts_tablet_manager.cc:1939] T d94e660ffee14d15ae22a11c6f43f049 P a21afbf8ee3e4b8991237a5bfdfcfcda: Deleting consensus metadata
I20260811 00:17:45.856551 10012 catalog_manager.cc:4841] TS a21afbf8ee3e4b8991237a5bfdfcfcda (127.7.94.3:43705): tablet d94e660ffee14d15ae22a11c6f43f049 (table test-table [id=876a21a942fd4006bcc28e8c9d0dc15b]) successfully deleted
I20260811 00:17:45.858479 10847 ts_tablet_manager.cc:1918] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 00:17:45.858646 10847 log.cc:1196] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2: Deleting WAL directory at /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786407133955383-7544-0/minicluster-data/ts-1/wal/wals/d94e660ffee14d15ae22a11c6f43f049
I20260811 00:17:45.859071 10847 ts_tablet_manager.cc:1939] T d94e660ffee14d15ae22a11c6f43f049 P c2418259ce0e4fe5863ff85ad593a1c2: Deleting consensus metadata
I20260811 00:17:45.860390 10012 catalog_manager.cc:4841] TS c2418259ce0e4fe5863ff85ad593a1c2 (127.7.94.2:43191): tablet d94e660ffee14d15ae22a11c6f43f049 (table test-table [id=876a21a942fd4006bcc28e8c9d0dc15b]) successfully deleted
I20260811 00:17:45.955153 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 10130
I20260811 00:17:45.980518 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 10352
W20260811 00:17:45.980518 10015 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.94.1:33363 (error 108)
I20260811 00:17:45.983482 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:45.982254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:33363 (local address 127.0.0.1:52540)
0811 00:17:45.983220 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:45.983311 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:33363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":851,"thread_start_us":760,"threads_started":1}
W20260811 00:17:45.983712 10015 catalog_manager.cc:4584] TS c2ef9800da9f4e4a937dee0a1cd7745b (127.7.94.1:33363): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d94e660ffee14d15ae22a11c6f43f049: Network error: Client connection negotiation failed: client connection to 127.7.94.1:33363: connect: Connection refused (error 111)
I20260811 00:17:45.988138 10800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:45.981392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41108 (local address 127.7.94.2:43191)
0811 00:17:45.981567 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:45.981571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:45.981593 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:17:45.981879 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:45.981884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:45.982000 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:45.982110 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:45.982999 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.983861 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.985073 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:45.985566 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:45.985711 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:45.986791 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:45.986807 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:45.986821 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:45.987372 (+ 551us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:45.987464 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:45.987492 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:45.987591 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:45.987719 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 00:17:45.987826 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 00:17:45.988828 10554 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:46.037127 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:46.036697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:33363 (local address 127.0.0.1:52556)
0811 00:17:46.036902 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:46.036985 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:33363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 00:17:46.038339 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:46.037952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:33363 (local address 127.0.0.1:52562)
0811 00:17:46.038118 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:46.038223 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:33363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 00:17:46.077590 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 10352
I20260811 00:17:46.099184 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:46.098556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:33363 (local address 127.0.0.1:52564)
0811 00:17:46.098873 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:46.098990 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:33363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204}
I20260811 00:17:46.100473 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:46.099979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:33363 (local address 127.0.0.1:52570)
0811 00:17:46.100202 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:46.100303 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:33363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260811 00:17:46.106590 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 10565
I20260811 00:17:46.117106 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:46.107423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37960 (local address 127.7.94.3:43705)
0811 00:17:46.107578 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:46.107583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:46.107622 (+ 39us) server_negotiation.cc:407] Connection header received
0811 00:17:46.108311 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:46.108317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:46.108428 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:46.108536 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:46.109001 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:46.109890 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:46.112093 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:46.112824 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:46.112962 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:46.114331 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:46.114340 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:46.114355 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:46.115067 (+ 712us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:46.115128 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:46.115152 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:46.115262 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:46.116700 (+ 1438us) server_negotiation.cc:299] Negotiation successful
0811 00:17:46.116785 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 00:17:46.117918 10760 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:46.181416 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:46.180776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:33363 (local address 127.0.0.1:52572)
0811 00:17:46.181023 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:46.181198 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:33363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 00:17:46.182531 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:46.182078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.1:33363 (local address 127.0.0.1:52582)
0811 00:17:46.182285 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 00:17:46.182359 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.1:33363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 00:17:46.209978 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 10565
I20260811 00:17:46.234994 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 9986
I20260811 00:17:46.236094 10086 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:46.345709 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 9986
2026-08-11T00:17:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3079 ms)
[----------] 6 tests from EncryptionPolicyTest (23076 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:17:46 dist-test-slave-8hhm krb5kdc[10865](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:17:46 dist-test-slave-8hhm krb5kdc[10865](info): set up 2 sockets
Aug 11 00:17:46 dist-test-slave-8hhm krb5kdc[10865](info): commencing operation
krb5kdc: starting...
W20260811 00:17:47.714468 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:17:47 dist-test-slave-8hhm krb5kdc[10865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:17:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:17:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:17:48.025182 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42447
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:17:48.360450 10888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:48.360877 10888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:48.361058 10888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:48.368173 10888 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:17:48.368324 10888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:48.368428 10888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:48.368535 10888 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:17:48.368600 10888 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:17:48.377909 10888 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42447
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:48.379166 10888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:48.380446 10888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:48.392354 10898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:48.392388 10894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:48.392443 10893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:48.393059 10888 server_base.cc:1034] running on GCE node
I20260811 00:17:48.393878 10888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:48.395222 10888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:48.396452 10888 hybrid_clock.cc:648] HybridClock initialized: now 1786407468396413 us; error 53 us; skew 500 ppm
I20260811 00:17:48.396797 10888 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:48 dist-test-slave-8hhm krb5kdc[10865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407468, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:48.399873 10888 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:17:48.401377 10888 webserver.cc:460] Webserver started at http://127.0.0.1:46793/ using document root <none> and password file <none>
I20260811 00:17:48.402050 10888 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:48.402194 10888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:48.402484 10888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:48.405375 10888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "010a57dd85c44501a93c9e42bf73ef3e"
format_stamp: "Formatted at 2026-08-11 00:17:48 on dist-test-slave-8hhm"
server_key: "4be26b465a5afb679e3e0d9d7f281793"
server_key_iv: "870d5eb9a3bb5a87f7d61ef3330cd049"
server_key_version: "encryptionkey@0"
I20260811 00:17:48.406127 10888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "010a57dd85c44501a93c9e42bf73ef3e"
format_stamp: "Formatted at 2026-08-11 00:17:48 on dist-test-slave-8hhm"
server_key: "4be26b465a5afb679e3e0d9d7f281793"
server_key_iv: "870d5eb9a3bb5a87f7d61ef3330cd049"
server_key_version: "encryptionkey@0"
I20260811 00:17:48.411190 10888 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 00:17:48.414505 10907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:48.415714 10888 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:17:48.415910 10888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "010a57dd85c44501a93c9e42bf73ef3e"
format_stamp: "Formatted at 2026-08-11 00:17:48 on dist-test-slave-8hhm"
server_key: "4be26b465a5afb679e3e0d9d7f281793"
server_key_iv: "870d5eb9a3bb5a87f7d61ef3330cd049"
server_key_version: "encryptionkey@0"
I20260811 00:17:48.416097 10888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:48.435146 10888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:48.439186 10888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:48.439460 10888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:48.457526 10888 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42447
I20260811 00:17:48.457556 10999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42447 every 8 connection(s)
I20260811 00:17:48.458838 10888 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:17:48.462468 11000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:48.465288 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 10888
I20260811 00:17:48.465517 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:17:48.466377 7544 external_mini_cluster.cc:1428] Setting key 61c8416c7070d14db41427b755023db9
I20260811 00:17:48.474218 11000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e: Bootstrap starting.
I20260811 00:17:48.477254 11000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:48.478372 11000 log.cc:824] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e: Log is configured to *not* fsync() on all Append() calls
Aug 11 00:17:48 dist-test-slave-8hhm krb5kdc[10865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 00:17:48.481647 11000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e: No bootstrap required, opened a new log
I20260811 00:17:48.489019 11005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:48.468285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34109 (local address 127.0.0.1:42447)
0811 00:17:48.469039 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:48.469051 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:48.469090 (+ 39us) server_negotiation.cc:407] Connection header received
0811 00:17:48.470760 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:48.470782 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:48.471145 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:48.471501 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:48.473006 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:48.474555 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:48.475942 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:48.476592 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:48.476787 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:48.481934 (+ 5147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:48.481942 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:48.481955 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:48.481988 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:48.485019 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:48.485906 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:48.485911 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:48.485917 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:48.486063 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:48.486495 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:48.486498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:48.486500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:48.486947 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:48.487187 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:48.487602 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 00:17:48.487883 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":332,"threads_started":1}
I20260811 00:17:48.490810 11000 raft_consensus.cc:348] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "010a57dd85c44501a93c9e42bf73ef3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42447 } }
I20260811 00:17:48.491202 11000 raft_consensus.cc:374] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:48.491314 11000 raft_consensus.cc:729] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 010a57dd85c44501a93c9e42bf73ef3e, State: Initialized, Role: FOLLOWER
I20260811 00:17:48.491990 11000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "010a57dd85c44501a93c9e42bf73ef3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42447 } }
I20260811 00:17:48.492251 11000 raft_consensus.cc:388] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:17:48.492419 11000 raft_consensus.cc:482] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:17:48.492599 11000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:48.493068 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:17:48.495770 11000 raft_consensus.cc:504] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "010a57dd85c44501a93c9e42bf73ef3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42447 } }
I20260811 00:17:48.496356 11000 leader_election.cc:302] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 010a57dd85c44501a93c9e42bf73ef3e; no voters:
I20260811 00:17:48.497089 11000 leader_election.cc:288] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:17:48.497181 11009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:17:48.497996 11009 raft_consensus.cc:686] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e [term 1 LEADER]: Becoming Leader. State: Replica: 010a57dd85c44501a93c9e42bf73ef3e, State: Running, Role: LEADER
I20260811 00:17:48.498687 11009 consensus_queue.cc:237] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "010a57dd85c44501a93c9e42bf73ef3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42447 } }
I20260811 00:17:48.500535 11000 sys_catalog.cc:564] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:17:48.501811 11014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 010a57dd85c44501a93c9e42bf73ef3e. Latest consensus state: current_term: 1 leader_uuid: "010a57dd85c44501a93c9e42bf73ef3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "010a57dd85c44501a93c9e42bf73ef3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42447 } } }
I20260811 00:17:48.501782 11015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "010a57dd85c44501a93c9e42bf73ef3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "010a57dd85c44501a93c9e42bf73ef3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42447 } } }
I20260811 00:17:48.502267 11015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:48.502267 11014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:48.503225 11021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:17:48.521054 11021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:17:48.522315 11005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:48.499441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42227 (local address 127.0.0.1:42447)
0811 00:17:48.499620 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:48.499625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:48.499841 (+ 216us) server_negotiation.cc:407] Connection header received
0811 00:17:48.500114 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:48.500137 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:48.500210 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:48.500321 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:48.502569 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:48.504132 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:48.506582 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:48.507593 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:48.507744 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:48.511819 (+ 4075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:48.511829 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:48.511836 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:48.511884 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:48.516524 (+ 4640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:48.517909 (+ 1385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:48.517922 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:48.517928 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:48.518118 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:48.518835 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:48.518844 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:48.518850 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:48.519354 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:48.520052 (+ 698us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:48.521500 (+ 1448us) server_negotiation.cc:299] Negotiation successful
0811 00:17:48.521741 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:17:48.531106 11021 catalog_manager.cc:1347] Generated new cluster ID: 5a0681ac48a74193a431a0ee3c28b4a0
I20260811 00:17:48.531248 11021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:17:48.549803 11021 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:17:48.551036 11021 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:17:48.561066 11021 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 010a57dd85c44501a93c9e42bf73ef3e: Generated new TSK 0
I20260811 00:17:48.562021 11021 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:17:48.575177 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 10888
I20260811 00:17:48.577095 10993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:48.666189 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 10888
W20260811 00:17:48.686725 10862 connection.cc:535] client connection to 127.0.0.1:42447 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42447): Connection reset by peer (error 104)
2026-08-11T00:17:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2304 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:17:48 dist-test-slave-8hhm krb5kdc[11057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:17:48 dist-test-slave-8hhm krb5kdc[11057](info): set up 2 sockets
Aug 11 00:17:48 dist-test-slave-8hhm krb5kdc[11057](info): commencing operation
krb5kdc: starting...
W20260811 00:17:50.015105 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.309s user 0.003s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:17:50 dist-test-slave-8hhm krb5kdc[11057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:17:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:17:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:17:50.322664 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36827
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:17:50.659968 11084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:50.660296 11084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:50.660465 11084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:50.667415 11084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:17:50.667549 11084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:50.667656 11084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:50.667758 11084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:17:50.667843 11084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:17:50.677290 11084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36827
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:50.678571 11084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:50.679925 11084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:50.691802 11101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:50.691893 11092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:50.691845 11094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:50.694535 11084 server_base.cc:1034] running on GCE node
I20260811 00:17:50.695348 11084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:50.696740 11084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:50.697962 11084 hybrid_clock.cc:648] HybridClock initialized: now 1786407470697929 us; error 43 us; skew 500 ppm
I20260811 00:17:50.698319 11084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:50 dist-test-slave-8hhm krb5kdc[11057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407470, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:50.706112 11084 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:17:50.707665 11084 webserver.cc:460] Webserver started at http://127.0.0.1:33521/ using document root <none> and password file <none>
I20260811 00:17:50.708410 11084 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:50.708564 11084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:50.708880 11084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:50.712005 11084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "8bf1d06656074954ad36741896ce54a2"
format_stamp: "Formatted at 2026-08-11 00:17:50 on dist-test-slave-8hhm"
server_key: "fd52c5ea612966c44d2e95d56dc37966"
server_key_iv: "2422f1a298de2125c223442bc22f7ff6"
server_key_version: "encryptionkey@0"
I20260811 00:17:50.712759 11084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "8bf1d06656074954ad36741896ce54a2"
format_stamp: "Formatted at 2026-08-11 00:17:50 on dist-test-slave-8hhm"
server_key: "fd52c5ea612966c44d2e95d56dc37966"
server_key_iv: "2422f1a298de2125c223442bc22f7ff6"
server_key_version: "encryptionkey@0"
I20260811 00:17:50.717831 11084 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 00:17:50.721159 11111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:50.722489 11084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:17:50.722675 11084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "8bf1d06656074954ad36741896ce54a2"
format_stamp: "Formatted at 2026-08-11 00:17:50 on dist-test-slave-8hhm"
server_key: "fd52c5ea612966c44d2e95d56dc37966"
server_key_iv: "2422f1a298de2125c223442bc22f7ff6"
server_key_version: "encryptionkey@0"
I20260811 00:17:50.722862 11084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:50.741143 11084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:50.745105 11084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:50.745412 11084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:50.764470 11084 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36827
I20260811 00:17:50.764503 11194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36827 every 8 connection(s)
I20260811 00:17:50.765873 11084 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:17:50.769627 11195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:50.771716 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 11084
I20260811 00:17:50.771915 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:17:50.772457 7544 external_mini_cluster.cc:1428] Setting key d778efc04b034cee6704bfff47e9534c
I20260811 00:17:50.784463 11195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2: Bootstrap starting.
I20260811 00:17:50.786369 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:50.774265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38787 (local address 127.0.0.1:36827)
0811 00:17:50.775141 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:50.775159 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:50.775200 (+ 41us) server_negotiation.cc:407] Connection header received
0811 00:17:50.776839 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:50.776867 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:50.777368 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:50.777715 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:50.778326 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:50.779617 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:50.781684 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:50.782384 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:50.782519 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:50.783958 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:50.783966 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:50.783981 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:50.784529 (+ 548us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:50.784605 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:50.784673 (+ 68us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:50.784816 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:50.785142 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 00:17:50.785376 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":432,"threads_started":1}
I20260811 00:17:50.788264 11195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:50.789660 11195 log.cc:824] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:50.790661 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:17:50.792646 11195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2: No bootstrap required, opened a new log
I20260811 00:17:50.802721 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:50.795488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47053 (local address 127.0.0.1:36827)
0811 00:17:50.795689 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:50.795696 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:50.796224 (+ 528us) server_negotiation.cc:407] Connection header received
0811 00:17:50.796407 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:50.796411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:50.796471 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:50.796553 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:50.797260 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:50.798051 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:50.800082 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:50.800543 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:50.800641 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:50.801878 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:50.801883 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:50.801886 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:50.801902 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:50.801937 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:50.801952 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:50.802032 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:50.802265 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 00:17:50.802307 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 00:17:50.802531 11195 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf1d06656074954ad36741896ce54a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36827 } }
I20260811 00:17:50.803040 11195 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:50.803162 11195 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bf1d06656074954ad36741896ce54a2, State: Initialized, Role: FOLLOWER
I20260811 00:17:50.803876 11195 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf1d06656074954ad36741896ce54a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36827 } }
I20260811 00:17:50.804179 11195 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:17:50.804347 11195 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:17:50.804536 11195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:50.806814 11195 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf1d06656074954ad36741896ce54a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36827 } }
I20260811 00:17:50.807309 11195 leader_election.cc:302] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bf1d06656074954ad36741896ce54a2; no voters:
I20260811 00:17:50.807889 11195 leader_election.cc:288] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:17:50.807986 11213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:17:50.809075 11213 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2 [term 1 LEADER]: Becoming Leader. State: Replica: 8bf1d06656074954ad36741896ce54a2, State: Running, Role: LEADER
I20260811 00:17:50.809790 11213 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf1d06656074954ad36741896ce54a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36827 } }
I20260811 00:17:50.810215 11195 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:17:50.812525 11214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bf1d06656074954ad36741896ce54a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf1d06656074954ad36741896ce54a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36827 } } }
I20260811 00:17:50.812948 11214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2 [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:50.813493 11216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bf1d06656074954ad36741896ce54a2. Latest consensus state: current_term: 1 leader_uuid: "8bf1d06656074954ad36741896ce54a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bf1d06656074954ad36741896ce54a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36827 } } }
I20260811 00:17:50.813833 11216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2 [sys.catalog]: This master's current role is: LEADER
W20260811 00:17:50.823405 11236 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:17:50.823570 11236 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:17:50.823758 11226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:17:50.826023 11226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:17:50.835186 11226 catalog_manager.cc:1347] Generated new cluster ID: 88a423b65d454ab090113fb6f557a701
I20260811 00:17:50.835307 11226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:17:50.881337 11226 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:17:50.883322 11226 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:17:50.893496 11226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8bf1d06656074954ad36741896ce54a2: Generated new TSK 0
I20260811 00:17:50.894345 11226 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:17:50.913775 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 11084
W20260811 00:17:50.914980 11189 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38787
W20260811 00:17:50.915920 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 11084: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:17:50.916115 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 11084
W20260811 00:17:50.942876 11055 connection.cc:535] client connection to 127.0.0.1:36827 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36827): Connection reset by peer (error 104)
2026-08-11T00:17:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2256 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:17:51 dist-test-slave-8hhm krb5kdc[11244](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:17:51 dist-test-slave-8hhm krb5kdc[11244](info): set up 2 sockets
Aug 11 00:17:51 dist-test-slave-8hhm krb5kdc[11244](info): commencing operation
krb5kdc: starting...
W20260811 00:17:52.270205 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:17:52 dist-test-slave-8hhm krb5kdc[11244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:17:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:17:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:17:52.578305 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42485
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42485
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:17:52.916891 11263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:52.917192 11263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:52.917368 11263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:52.924326 11263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:17:52.924430 11263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:52.924484 11263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:52.924544 11263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:17:52.924588 11263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:17:52.934790 11263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42485
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:52.936075 11263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:52.937409 11263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:52.949532 11271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:52.949581 11272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:52.949914 11275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:52.950680 11263 server_base.cc:1034] running on GCE node
I20260811 00:17:52.951606 11263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:52.953053 11263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:52.954263 11263 hybrid_clock.cc:648] HybridClock initialized: now 1786407472954202 us; error 53 us; skew 500 ppm
I20260811 00:17:52.954610 11263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:52 dist-test-slave-8hhm krb5kdc[11244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407472, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:52.957681 11263 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:17:52.959054 11263 webserver.cc:460] Webserver started at http://127.0.0.1:43417/ using document root <none> and password file <none>
I20260811 00:17:52.959671 11263 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:52.959767 11263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:52.960003 11263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:52.962812 11263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "2c7883a7e5254e8a9ead2c375b109c13"
format_stamp: "Formatted at 2026-08-11 00:17:52 on dist-test-slave-8hhm"
server_key: "7e74e13b544dfda496670e64443301b4"
server_key_iv: "445635519a3bc162181d79b4f8d57d7a"
server_key_version: "encryptionkey@0"
I20260811 00:17:52.963510 11263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "2c7883a7e5254e8a9ead2c375b109c13"
format_stamp: "Formatted at 2026-08-11 00:17:52 on dist-test-slave-8hhm"
server_key: "7e74e13b544dfda496670e64443301b4"
server_key_iv: "445635519a3bc162181d79b4f8d57d7a"
server_key_version: "encryptionkey@0"
I20260811 00:17:52.968250 11263 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 00:17:52.971541 11291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:52.972826 11263 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 00:17:52.973009 11263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "2c7883a7e5254e8a9ead2c375b109c13"
format_stamp: "Formatted at 2026-08-11 00:17:52 on dist-test-slave-8hhm"
server_key: "7e74e13b544dfda496670e64443301b4"
server_key_iv: "445635519a3bc162181d79b4f8d57d7a"
server_key_version: "encryptionkey@0"
I20260811 00:17:52.973198 11263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:52.978538 11263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:52.982342 11263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:52.982637 11263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:53.000469 11263 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42485
I20260811 00:17:53.000504 11382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42485 every 8 connection(s)
I20260811 00:17:53.001884 11263 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:17:53.005686 11383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:53.007766 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 11263
I20260811 00:17:53.007943 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:17:53.008411 7544 external_mini_cluster.cc:1428] Setting key 545ecb117e67d78ebc4d244e6e192b9e
I20260811 00:17:53.017738 11388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:53.010401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60361 (local address 127.0.0.1:42485)
0811 00:17:53.011291 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:53.011310 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:53.011365 (+ 55us) server_negotiation.cc:407] Connection header received
0811 00:17:53.013855 (+ 2490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:53.013885 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:53.014304 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:53.014765 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:53.015748 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:53.015756 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:53.015772 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:53.015798 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:53.015882 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:53.015908 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:53.016125 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:53.016257 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 00:17:53.016471 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":358,"threads_started":1}
I20260811 00:17:53.019572 11383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13: Bootstrap starting.
I20260811 00:17:53.021579 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:17:53.023224 11383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:53.024489 11383 log.cc:824] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:53.027634 11383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13: No bootstrap required, opened a new log
I20260811 00:17:53.029527 11388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:53.026928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41377 (local address 127.0.0.1:42485)
0811 00:17:53.027103 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:53.027111 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:53.027442 (+ 331us) server_negotiation.cc:407] Connection header received
0811 00:17:53.027638 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:53.027642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:53.027687 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:53.027795 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:53.028783 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:53.028794 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:53.028798 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:53.028815 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:53.028851 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:53.028864 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:53.028979 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:53.029182 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 00:17:53.029269 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 00:17:53.036388 11383 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c7883a7e5254e8a9ead2c375b109c13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42485 } }
I20260811 00:17:53.036794 11383 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:53.036906 11383 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7883a7e5254e8a9ead2c375b109c13, State: Initialized, Role: FOLLOWER
I20260811 00:17:53.037580 11383 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c7883a7e5254e8a9ead2c375b109c13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42485 } }
I20260811 00:17:53.037825 11383 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:17:53.038000 11383 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:17:53.038189 11383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:53.040709 11383 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c7883a7e5254e8a9ead2c375b109c13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42485 } }
I20260811 00:17:53.041141 11383 leader_election.cc:302] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c7883a7e5254e8a9ead2c375b109c13; no voters:
I20260811 00:17:53.041719 11383 leader_election.cc:288] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:17:53.041858 11399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:17:53.042538 11399 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7883a7e5254e8a9ead2c375b109c13, State: Running, Role: LEADER
I20260811 00:17:53.043058 11399 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c7883a7e5254e8a9ead2c375b109c13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42485 } }
I20260811 00:17:53.043879 11383 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:17:53.046070 11400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c7883a7e5254e8a9ead2c375b109c13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c7883a7e5254e8a9ead2c375b109c13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42485 } } }
I20260811 00:17:53.046787 11400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13 [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:53.046263 11404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c7883a7e5254e8a9ead2c375b109c13. Latest consensus state: current_term: 1 leader_uuid: "2c7883a7e5254e8a9ead2c375b109c13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c7883a7e5254e8a9ead2c375b109c13" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42485 } } }
I20260811 00:17:53.049465 11404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13 [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:53.050844 11415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:17:53.055946 11415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:17:53.064672 11415 catalog_manager.cc:1347] Generated new cluster ID: 6eec87cab94841e584faf382a64c98b7
I20260811 00:17:53.064800 11415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:17:53.085279 11415 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:17:53.085726 11415 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:17:53.095253 11415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c7883a7e5254e8a9ead2c375b109c13: Generated new TSK 0
I20260811 00:17:53.096231 11415 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:17:53.112671 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 11263
W20260811 00:17:53.113682 11378 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60361
W20260811 00:17:53.114346 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 11263: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:17:53.114498 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 11263
2026-08-11T00:17:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2199 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:17:53 dist-test-slave-8hhm krb5kdc[11437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:17:53 dist-test-slave-8hhm krb5kdc[11437](info): set up 2 sockets
Aug 11 00:17:53 dist-test-slave-8hhm krb5kdc[11437](info): commencing operation
krb5kdc: starting...
W20260811 00:17:54.498770 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.337s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:17:54 dist-test-slave-8hhm krb5kdc[11437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:17:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:17:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:17:54.816783 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41143
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:17:55.159437 11463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:55.159714 11463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:55.159915 11463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:55.166766 11463 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:17:55.166951 11463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:55.167086 11463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:55.167200 11463 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:17:55.167270 11463 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:17:55.177443 11463 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:55.178874 11463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:55.180246 11463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:55.192261 11472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:55.192314 11474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:55.192233 11470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:55.193553 11463 server_base.cc:1034] running on GCE node
I20260811 00:17:55.194358 11463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:55.195696 11463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:55.196941 11463 hybrid_clock.cc:648] HybridClock initialized: now 1786407475196901 us; error 51 us; skew 500 ppm
I20260811 00:17:55.197366 11463 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:55 dist-test-slave-8hhm krb5kdc[11437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407475, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:55.200452 11463 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:17:55.201880 11463 webserver.cc:460] Webserver started at http://127.0.0.1:36347/ using document root <none> and password file <none>
I20260811 00:17:55.202564 11463 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:55.202716 11463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:55.203019 11463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:55.206027 11463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "589cc1f6cee14d678dc0bf9aa89d881f"
format_stamp: "Formatted at 2026-08-11 00:17:55 on dist-test-slave-8hhm"
server_key: "5d96dab96769126114451c089f0f5202"
server_key_iv: "7e1c4023f06a122ab190bac166b33715"
server_key_version: "encryptionkey@0"
I20260811 00:17:55.206769 11463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "589cc1f6cee14d678dc0bf9aa89d881f"
format_stamp: "Formatted at 2026-08-11 00:17:55 on dist-test-slave-8hhm"
server_key: "5d96dab96769126114451c089f0f5202"
server_key_iv: "7e1c4023f06a122ab190bac166b33715"
server_key_version: "encryptionkey@0"
I20260811 00:17:55.211983 11463 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:17:55.215370 11491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:55.216607 11463 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 00:17:55.216782 11463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "589cc1f6cee14d678dc0bf9aa89d881f"
format_stamp: "Formatted at 2026-08-11 00:17:55 on dist-test-slave-8hhm"
server_key: "5d96dab96769126114451c089f0f5202"
server_key_iv: "7e1c4023f06a122ab190bac166b33715"
server_key_version: "encryptionkey@0"
I20260811 00:17:55.216971 11463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:55.250586 11463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:55.254976 11463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:55.255272 11463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:55.274184 11463 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41143
I20260811 00:17:55.274215 11571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41143 every 8 connection(s)
I20260811 00:17:55.275714 11463 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:17:55.278741 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 11463
I20260811 00:17:55.278931 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:17:55.279469 7544 external_mini_cluster.cc:1428] Setting key 77bcf0934d43384b3e6f3622b5257828
I20260811 00:17:55.279976 11573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:55.294440 11573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f: Bootstrap starting.
Aug 11 00:17:55 dist-test-slave-8hhm krb5kdc[11437](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 00:17:55.297693 11573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:55.298787 11573 log.cc:824] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:55.301731 11573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f: No bootstrap required, opened a new log
I20260811 00:17:55.303609 11576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:55.281405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36849 (local address 127.0.0.1:41143)
0811 00:17:55.282194 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:55.282205 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:55.282245 (+ 40us) server_negotiation.cc:407] Connection header received
0811 00:17:55.284523 (+ 2278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:55.284544 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:55.284927 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:55.285311 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:55.287036 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:55.288305 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:55.290198 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:55.290880 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:55.291028 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:55.296112 (+ 5084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:55.296121 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:55.296136 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:55.296175 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:55.299456 (+ 3281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:55.300229 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:55.300235 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:55.300241 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:55.300408 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:55.300832 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:55.300836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:55.300838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:55.301442 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:55.301765 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:55.302219 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 00:17:55.302620 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":373,"threads_started":1}
I20260811 00:17:55.307386 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:17:55.310070 11573 raft_consensus.cc:348] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589cc1f6cee14d678dc0bf9aa89d881f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41143 } }
I20260811 00:17:55.310527 11573 raft_consensus.cc:374] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:55.310642 11573 raft_consensus.cc:729] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 589cc1f6cee14d678dc0bf9aa89d881f, State: Initialized, Role: FOLLOWER
I20260811 00:17:55.311309 11573 consensus_queue.cc:260] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589cc1f6cee14d678dc0bf9aa89d881f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41143 } }
I20260811 00:17:55.311534 11573 raft_consensus.cc:388] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:17:55.311691 11573 raft_consensus.cc:482] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:17:55.311911 11573 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:55.315356 11573 raft_consensus.cc:504] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589cc1f6cee14d678dc0bf9aa89d881f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41143 } }
I20260811 00:17:55.315961 11573 leader_election.cc:302] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 589cc1f6cee14d678dc0bf9aa89d881f; no voters:
I20260811 00:17:55.316646 11573 leader_election.cc:288] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:17:55.316720 11590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:17:55.318112 11590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f [term 1 LEADER]: Becoming Leader. State: Replica: 589cc1f6cee14d678dc0bf9aa89d881f, State: Running, Role: LEADER
I20260811 00:17:55.318935 11573 sys_catalog.cc:564] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:17:55.318823 11590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589cc1f6cee14d678dc0bf9aa89d881f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41143 } }
I20260811 00:17:55.326037 11594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "589cc1f6cee14d678dc0bf9aa89d881f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589cc1f6cee14d678dc0bf9aa89d881f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41143 } } }
I20260811 00:17:55.326921 11594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:55.326462 11596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 589cc1f6cee14d678dc0bf9aa89d881f. Latest consensus state: current_term: 1 leader_uuid: "589cc1f6cee14d678dc0bf9aa89d881f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589cc1f6cee14d678dc0bf9aa89d881f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41143 } } }
I20260811 00:17:55.327220 11596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:55.332376 11576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:55.312064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42911 (local address 127.0.0.1:41143)
0811 00:17:55.312276 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:55.312285 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:55.312724 (+ 439us) server_negotiation.cc:407] Connection header received
0811 00:17:55.312890 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:55.312894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:55.312961 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:55.313051 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:55.314894 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:55.315636 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:55.317583 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:55.318293 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:55.318485 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:55.322343 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:55.322353 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:55.322361 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:17:55.322413 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:55.328387 (+ 5974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:55.329522 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:55.329530 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:55.329536 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:55.329758 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:17:55.330331 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:17:55.330338 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:17:55.330343 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:17:55.330796 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:55.331122 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:55.331532 (+ 410us) server_negotiation.cc:299] Negotiation successful
0811 00:17:55.331848 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 00:17:55.338706 11605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:17:55.344501 11605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:17:55.355051 11605 catalog_manager.cc:1347] Generated new cluster ID: 0887a462a9154d1c9465fd197579df9d
I20260811 00:17:55.355219 11605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:17:55.368989 11605 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:17:55.370273 11605 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:17:55.378964 11605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 589cc1f6cee14d678dc0bf9aa89d881f: Generated new TSK 0
I20260811 00:17:55.379743 11605 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:17:55.391103 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 11463
I20260811 00:17:55.392168 11564 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:17:55.482055 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 11463
2026-08-11T00:17:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2366 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:17:55 dist-test-slave-8hhm krb5kdc[11626](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:17:55 dist-test-slave-8hhm krb5kdc[11626](info): set up 2 sockets
Aug 11 00:17:55 dist-test-slave-8hhm krb5kdc[11626](info): commencing operation
krb5kdc: starting...
W20260811 00:17:56.859297 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.331s user 0.003s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:17:57 dist-test-slave-8hhm krb5kdc[11626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:17:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:17:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:17:57.185995 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35553
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:17:57.532142 11649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:57.532493 11649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:57.532673 11649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:57.539822 11649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:17:57.540020 11649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:57.540130 11649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:57.540277 11649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:17:57.540357 11649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:17:57.550486 11649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:57.551841 11649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:57.553228 11649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:57.565760 11654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:57.565704 11655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:57.565733 11658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:57.567842 11649 server_base.cc:1034] running on GCE node
I20260811 00:17:57.568735 11649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:57.570217 11649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:57.571475 11649 hybrid_clock.cc:648] HybridClock initialized: now 1786407477571398 us; error 38 us; skew 500 ppm
I20260811 00:17:57.571902 11649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:57 dist-test-slave-8hhm krb5kdc[11626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407477, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:57.575093 11649 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:17:57.576483 11649 webserver.cc:460] Webserver started at http://127.0.0.1:41795/ using document root <none> and password file <none>
I20260811 00:17:57.577134 11649 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:57.577287 11649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:57.577540 11649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:57.580413 11649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "34ee5838d30346998b7cbec929ea1392"
format_stamp: "Formatted at 2026-08-11 00:17:57 on dist-test-slave-8hhm"
server_key: "a36efef76532803f7ab6dd69848ead7e"
server_key_iv: "9c7d8abef31b9cccb929885699a866ea"
server_key_version: "encryptionkey@0"
I20260811 00:17:57.581112 11649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "34ee5838d30346998b7cbec929ea1392"
format_stamp: "Formatted at 2026-08-11 00:17:57 on dist-test-slave-8hhm"
server_key: "a36efef76532803f7ab6dd69848ead7e"
server_key_iv: "9c7d8abef31b9cccb929885699a866ea"
server_key_version: "encryptionkey@0"
I20260811 00:17:57.586185 11649 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 00:17:57.589514 11667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:57.590788 11649 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:17:57.590965 11649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "34ee5838d30346998b7cbec929ea1392"
format_stamp: "Formatted at 2026-08-11 00:17:57 on dist-test-slave-8hhm"
server_key: "a36efef76532803f7ab6dd69848ead7e"
server_key_iv: "9c7d8abef31b9cccb929885699a866ea"
server_key_version: "encryptionkey@0"
I20260811 00:17:57.591152 11649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:57.628535 11649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:57.633069 11649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:57.633399 11649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:57.651726 11649 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35553
I20260811 00:17:57.651751 11749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35553 every 8 connection(s)
I20260811 00:17:57.653137 11649 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:17:57.656966 11750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:57.659247 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 11649
I20260811 00:17:57.659458 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:17:57.659921 7544 external_mini_cluster.cc:1428] Setting key 8944d4dd4f18aa15509cf743aea48754
I20260811 00:17:57.670501 11750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392: Bootstrap starting.
I20260811 00:17:57.674319 11750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:57.675146 11753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:57.661808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39623 (local address 127.0.0.1:35553)
0811 00:17:57.662589 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:57.662611 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:57.662664 (+ 53us) server_negotiation.cc:407] Connection header received
0811 00:17:57.665124 (+ 2460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:57.665154 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:57.665674 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:57.666127 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:57.666644 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:57.668712 (+ 2068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:57.670023 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:57.671044 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:57.671230 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:57.672339 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:57.672349 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:57.672367 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:57.673073 (+ 706us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:57.673179 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:57.673228 (+ 49us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:57.673354 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:57.673532 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 00:17:57.673782 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":334,"threads_started":1}
I20260811 00:17:57.675673 11750 log.cc:824] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:57.678632 11750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392: No bootstrap required, opened a new log
I20260811 00:17:57.678889 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:17:57.686978 11750 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34ee5838d30346998b7cbec929ea1392" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35553 } }
I20260811 00:17:57.687435 11750 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:57.687551 11750 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34ee5838d30346998b7cbec929ea1392, State: Initialized, Role: FOLLOWER
I20260811 00:17:57.688205 11750 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34ee5838d30346998b7cbec929ea1392" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35553 } }
I20260811 00:17:57.688458 11750 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:17:57.688607 11750 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:17:57.688824 11750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:57.689850 11753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:57.683237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37389 (local address 127.0.0.1:35553)
0811 00:17:57.683416 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:57.683422 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:57.683791 (+ 369us) server_negotiation.cc:407] Connection header received
0811 00:17:57.683932 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:57.683937 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:57.684013 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:57.684137 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:57.684646 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:57.685807 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:57.687163 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:17:57.687810 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:17:57.687928 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:17:57.689043 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:57.689049 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:57.689055 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:57.689077 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:57.689127 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:57.689160 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:57.689293 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:57.689426 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 00:17:57.689520 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 00:17:57.691396 11750 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34ee5838d30346998b7cbec929ea1392" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35553 } }
I20260811 00:17:57.691867 11750 leader_election.cc:302] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34ee5838d30346998b7cbec929ea1392; no voters:
I20260811 00:17:57.692426 11750 leader_election.cc:288] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:17:57.692545 11765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:17:57.693377 11765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392 [term 1 LEADER]: Becoming Leader. State: Replica: 34ee5838d30346998b7cbec929ea1392, State: Running, Role: LEADER
I20260811 00:17:57.693954 11765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34ee5838d30346998b7cbec929ea1392" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35553 } }
I20260811 00:17:57.694636 11750 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:17:57.696874 11771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34ee5838d30346998b7cbec929ea1392. Latest consensus state: current_term: 1 leader_uuid: "34ee5838d30346998b7cbec929ea1392" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34ee5838d30346998b7cbec929ea1392" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35553 } } }
I20260811 00:17:57.697357 11771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392 [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:57.698249 11770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34ee5838d30346998b7cbec929ea1392" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34ee5838d30346998b7cbec929ea1392" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35553 } } }
I20260811 00:17:57.698752 11770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392 [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:57.698957 11779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:17:57.705431 11779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:17:57.714812 11779 catalog_manager.cc:1347] Generated new cluster ID: 9508d7698c974a9f929354fd0b9a932d
I20260811 00:17:57.714931 11779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:17:57.730940 11779 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:17:57.732192 11779 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:17:57.738299 11779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34ee5838d30346998b7cbec929ea1392: Generated new TSK 0
I20260811 00:17:57.739055 11779 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:17:57.747213 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 11649
W20260811 00:17:57.748414 11744 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39623
W20260811 00:17:57.749167 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 11649: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:17:57.749342 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 11649
2026-08-11T00:17:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2271 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:17:57 dist-test-slave-8hhm krb5kdc[11799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:17:57 dist-test-slave-8hhm krb5kdc[11799](info): set up 2 sockets
Aug 11 00:17:57 dist-test-slave-8hhm krb5kdc[11799](info): commencing operation
krb5kdc: starting...
W20260811 00:17:59.152663 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.353s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:17:59 dist-test-slave-8hhm krb5kdc[11799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:17:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:17:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:17:59.472371 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38383
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38383
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:17:59.811645 11818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:17:59.811977 11818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:17:59.812180 11818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:17:59.819376 11818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:17:59.819502 11818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:17:59.819548 11818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:17:59.819643 11818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:17:59.819715 11818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:17:59.829501 11818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:17:59.830790 11818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:17:59.832181 11818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:17:59.844334 11828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:59.844390 11825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:17:59.844372 11826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:17:59.845278 11818 server_base.cc:1034] running on GCE node
I20260811 00:17:59.846143 11818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:17:59.847616 11818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:17:59.848847 11818 hybrid_clock.cc:648] HybridClock initialized: now 1786407479848795 us; error 46 us; skew 500 ppm
I20260811 00:17:59.849202 11818 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:17:59 dist-test-slave-8hhm krb5kdc[11799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407479, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:17:59.852495 11818 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:17:59.853964 11818 webserver.cc:460] Webserver started at http://127.0.0.1:44577/ using document root <none> and password file <none>
I20260811 00:17:59.854640 11818 fs_manager.cc:359] Metadata directory not provided
I20260811 00:17:59.854740 11818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:17:59.855003 11818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:17:59.857960 11818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "3589644685ab49cba7777925845da52c"
format_stamp: "Formatted at 2026-08-11 00:17:59 on dist-test-slave-8hhm"
server_key: "efe9b5b4bcb9c9520d53d9b5b9036a9c"
server_key_iv: "ff517d8e1afba10b37e9fd8807a245ae"
server_key_version: "encryptionkey@0"
I20260811 00:17:59.858690 11818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "3589644685ab49cba7777925845da52c"
format_stamp: "Formatted at 2026-08-11 00:17:59 on dist-test-slave-8hhm"
server_key: "efe9b5b4bcb9c9520d53d9b5b9036a9c"
server_key_iv: "ff517d8e1afba10b37e9fd8807a245ae"
server_key_version: "encryptionkey@0"
I20260811 00:17:59.863756 11818 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:17:59.867270 11839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:17:59.868919 11818 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:17:59.869125 11818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "3589644685ab49cba7777925845da52c"
format_stamp: "Formatted at 2026-08-11 00:17:59 on dist-test-slave-8hhm"
server_key: "efe9b5b4bcb9c9520d53d9b5b9036a9c"
server_key_iv: "ff517d8e1afba10b37e9fd8807a245ae"
server_key_version: "encryptionkey@0"
I20260811 00:17:59.869396 11818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:17:59.875038 11818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:17:59.879047 11818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:17:59.879365 11818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:17:59.897984 11818 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38383
I20260811 00:17:59.898102 11925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38383 every 8 connection(s)
I20260811 00:17:59.899477 11818 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:17:59.902580 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 11818
I20260811 00:17:59.902793 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:17:59.903374 7544 external_mini_cluster.cc:1428] Setting key c5c39f9e9693e3782779f39f932940b6
I20260811 00:17:59.903537 11928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:17:59.911274 11934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:59.905123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44883 (local address 127.0.0.1:38383)
0811 00:17:59.905962 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:59.905973 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:59.906043 (+ 70us) server_negotiation.cc:407] Connection header received
0811 00:17:59.908079 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:59.908100 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:59.908414 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:59.908743 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:59.909817 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:59.909822 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:59.909834 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:59.909847 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:59.909904 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:59.909925 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:59.910088 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:59.910193 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 00:17:59.910393 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":369,"threads_started":1}
I20260811 00:17:59.914793 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:17:59.917026 11928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c: Bootstrap starting.
I20260811 00:17:59.921173 11928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c: Neither blocks nor log segments found. Creating new log.
I20260811 00:17:59.922564 11928 log.cc:824] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c: Log is configured to *not* fsync() on all Append() calls
I20260811 00:17:59.922791 11934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:17:59.920018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48071 (local address 127.0.0.1:38383)
0811 00:17:59.920173 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 00:17:59.920177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:17:59.920581 (+ 404us) server_negotiation.cc:407] Connection header received
0811 00:17:59.920839 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:17:59.920843 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:17:59.920888 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:17:59.920963 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:17:59.921997 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:17:59.922002 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:17:59.922005 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:17:59.922022 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:17:59.922066 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:17:59.922083 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:17:59.922189 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 00:17:59.922338 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 00:17:59.922382 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 00:17:59.925441 11928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c: No bootstrap required, opened a new log
I20260811 00:17:59.932663 11928 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3589644685ab49cba7777925845da52c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38383 } }
I20260811 00:17:59.932994 11928 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:17:59.933118 11928 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3589644685ab49cba7777925845da52c, State: Initialized, Role: FOLLOWER
I20260811 00:17:59.933765 11928 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3589644685ab49cba7777925845da52c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38383 } }
I20260811 00:17:59.933985 11928 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:17:59.934137 11928 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:17:59.934322 11928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:17:59.936683 11928 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3589644685ab49cba7777925845da52c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38383 } }
I20260811 00:17:59.937147 11928 leader_election.cc:302] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3589644685ab49cba7777925845da52c; no voters:
I20260811 00:17:59.937786 11928 leader_election.cc:288] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:17:59.937934 11945 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:17:59.938668 11945 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c [term 1 LEADER]: Becoming Leader. State: Replica: 3589644685ab49cba7777925845da52c, State: Running, Role: LEADER
I20260811 00:17:59.939282 11945 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3589644685ab49cba7777925845da52c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38383 } }
I20260811 00:17:59.939896 11928 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:17:59.942330 11947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3589644685ab49cba7777925845da52c. Latest consensus state: current_term: 1 leader_uuid: "3589644685ab49cba7777925845da52c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3589644685ab49cba7777925845da52c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38383 } } }
I20260811 00:17:59.942410 11946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3589644685ab49cba7777925845da52c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3589644685ab49cba7777925845da52c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38383 } } }
I20260811 00:17:59.943125 11947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:59.943207 11946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c [sys.catalog]: This master's current role is: LEADER
I20260811 00:17:59.951236 11966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:17:59.955727 11966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:17:59.963945 11966 catalog_manager.cc:1347] Generated new cluster ID: 8e4991c30e5c460a84c3140d7080fe4d
I20260811 00:17:59.964062 11966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:17:59.985450 11966 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:17:59.985991 11966 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:17:59.998564 11966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3589644685ab49cba7777925845da52c: Generated new TSK 0
I20260811 00:17:59.999531 11966 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:18:00.009033 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 11818
W20260811 00:18:00.010179 11919 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44883
W20260811 00:18:00.010885 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 11818: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:18:00.011044 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 11818
2026-08-11T00:18:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2259 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:18:00 dist-test-slave-8hhm krb5kdc[11980](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:18:00 dist-test-slave-8hhm krb5kdc[11980](info): set up 2 sockets
Aug 11 00:18:00 dist-test-slave-8hhm krb5kdc[11980](info): commencing operation
krb5kdc: starting...
W20260811 00:18:01.394677 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.334s user 0.002s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:18:01 dist-test-slave-8hhm krb5kdc[11980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:18:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:18:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:18:01.717976 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40939
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:02.070211 11998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:02.070564 11998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:02.070737 11998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:02.078053 11998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:02.078197 11998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:02.078312 11998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:02.078424 11998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:02.078502 11998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:02.088263 11998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:02.089607 11998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:02.090911 11998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:02.102949 12006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:02.102967 12007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:02.103008 12010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:02.103848 11998 server_base.cc:1034] running on GCE node
I20260811 00:18:02.104624 11998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:02.106164 11998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:02.107416 11998 hybrid_clock.cc:648] HybridClock initialized: now 1786407482107340 us; error 63 us; skew 500 ppm
I20260811 00:18:02.107834 11998 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:02 dist-test-slave-8hhm krb5kdc[11980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407482, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:02.116083 11998 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:18:02.117676 11998 webserver.cc:460] Webserver started at http://127.0.0.1:45617/ using document root <none> and password file <none>
I20260811 00:18:02.118332 11998 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:02.118429 11998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:02.118676 11998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:02.121546 11998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "9939c99b71514d9eb8d564906a3fc814"
format_stamp: "Formatted at 2026-08-11 00:18:02 on dist-test-slave-8hhm"
server_key: "6d1ca1b39e516c8c5850fec7202db94c"
server_key_iv: "a7a6c28f47d0998d1b163a7f02d1555f"
server_key_version: "encryptionkey@0"
I20260811 00:18:02.122249 11998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "9939c99b71514d9eb8d564906a3fc814"
format_stamp: "Formatted at 2026-08-11 00:18:02 on dist-test-slave-8hhm"
server_key: "6d1ca1b39e516c8c5850fec7202db94c"
server_key_iv: "a7a6c28f47d0998d1b163a7f02d1555f"
server_key_version: "encryptionkey@0"
I20260811 00:18:02.127350 11998 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:18:02.130681 12023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:02.131970 11998 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 00:18:02.132169 11998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "9939c99b71514d9eb8d564906a3fc814"
format_stamp: "Formatted at 2026-08-11 00:18:02 on dist-test-slave-8hhm"
server_key: "6d1ca1b39e516c8c5850fec7202db94c"
server_key_iv: "a7a6c28f47d0998d1b163a7f02d1555f"
server_key_version: "encryptionkey@0"
I20260811 00:18:02.132404 11998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:02.153110 11998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:02.157347 11998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:02.157655 11998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:02.176584 11998 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40939
I20260811 00:18:02.176591 12111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40939 every 8 connection(s)
I20260811 00:18:02.178035 11998 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:18:02.181689 12112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:02.187834 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 11998
I20260811 00:18:02.188031 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:18:02.188568 7544 external_mini_cluster.cc:1428] Setting key 47368b99b47b46a6727ad4ed0a079366
I20260811 00:18:02.192714 12112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814: Bootstrap starting.
I20260811 00:18:02.196465 12112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:02.197685 12112 log.cc:824] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:02.200831 12112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814: No bootstrap required, opened a new log
Aug 11 00:18:02 dist-test-slave-8hhm krb5kdc[11980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 00:18:02.209121 12112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9939c99b71514d9eb8d564906a3fc814" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40939 } }
I20260811 00:18:02.209584 12112 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:18:02.209709 12112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9939c99b71514d9eb8d564906a3fc814, State: Initialized, Role: FOLLOWER
I20260811 00:18:02.210311 12112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9939c99b71514d9eb8d564906a3fc814" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40939 } }
I20260811 00:18:02.210561 12112 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:18:02.210742 12112 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:18:02.210953 12112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:02.212416 12115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:02.190529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:36227 (local address 127.0.0.1:40939)
0811 00:18:02.191305 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:02.191316 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:02.191358 (+ 42us) server_negotiation.cc:407] Connection header received
0811 00:18:02.193071 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:02.193092 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:02.193548 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:02.193855 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:02.195663 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:02.197014 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:02.198803 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:02.199473 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:02.199597 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:02.204495 (+ 4898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:02.204502 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:02.204517 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:02.204563 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:02.207901 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:02.208795 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:02.208802 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:02.208809 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:02.208980 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:02.209496 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:02.209501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:02.209503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:02.210214 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:02.210555 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:02.211035 (+ 480us) server_negotiation.cc:299] Negotiation successful
0811 00:18:02.211373 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":370,"threads_started":1}
I20260811 00:18:02.213512 12112 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9939c99b71514d9eb8d564906a3fc814" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40939 } }
I20260811 00:18:02.214036 12112 leader_election.cc:302] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9939c99b71514d9eb8d564906a3fc814; no voters:
I20260811 00:18:02.214774 12112 leader_election.cc:288] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:18:02.214892 12121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:18:02.216312 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:18:02.217383 12112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:18:02.217482 12121 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814 [term 1 LEADER]: Becoming Leader. State: Replica: 9939c99b71514d9eb8d564906a3fc814, State: Running, Role: LEADER
I20260811 00:18:02.218236 12121 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9939c99b71514d9eb8d564906a3fc814" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40939 } }
I20260811 00:18:02.223048 12123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9939c99b71514d9eb8d564906a3fc814" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9939c99b71514d9eb8d564906a3fc814" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40939 } } }
I20260811 00:18:02.223562 12123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:02.224184 12124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9939c99b71514d9eb8d564906a3fc814. Latest consensus state: current_term: 1 leader_uuid: "9939c99b71514d9eb8d564906a3fc814" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9939c99b71514d9eb8d564906a3fc814" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40939 } } }
I20260811 00:18:02.224543 12124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:02.227211 12135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:18:02.233566 12135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:18:02.250545 12135 catalog_manager.cc:1347] Generated new cluster ID: 573c6fa1ddaa43599ff48dfeef4821e0
I20260811 00:18:02.250816 12135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:18:02.253239 12115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:02.228199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:59779 (local address 127.0.0.1:40939)
0811 00:18:02.228416 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:02.228426 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:02.229619 (+ 1193us) server_negotiation.cc:407] Connection header received
0811 00:18:02.229952 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:02.229960 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:02.230066 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:02.230164 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:02.232999 (+ 2835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:02.234663 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:02.237477 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:02.238578 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:02.238742 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:02.243346 (+ 4604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:02.243356 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:02.243365 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:02.243425 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:02.249118 (+ 5693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:02.250295 (+ 1177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:02.250300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:02.250304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:02.250478 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:02.251151 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:02.251156 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:02.251159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:02.251588 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:02.251929 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:02.252645 (+ 716us) server_negotiation.cc:299] Negotiation successful
0811 00:18:02.252866 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 00:18:02.266783 12135 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:18:02.268159 12135 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:18:02.280278 12135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9939c99b71514d9eb8d564906a3fc814: Generated new TSK 0
I20260811 00:18:02.281119 12135 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:18:02.292783 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 11998
I20260811 00:18:02.294620 12104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:18:02.385344 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 11998
W20260811 00:18:02.413355 11977 connection.cc:535] client connection to 127.0.0.1:40939 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40939): Connection reset by peer (error 104)
2026-08-11T00:18:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2372 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:18:02 dist-test-slave-8hhm krb5kdc[12164](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:18:02 dist-test-slave-8hhm krb5kdc[12164](info): set up 2 sockets
Aug 11 00:18:02 dist-test-slave-8hhm krb5kdc[12164](info): commencing operation
krb5kdc: starting...
W20260811 00:18:03.772557 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.340s user 0.003s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:18:03 dist-test-slave-8hhm krb5kdc[12164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407483, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:18:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:18:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:18:04.102589 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39965
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:04.446072 12187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:04.446369 12187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:04.446473 12187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:04.453603 12187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:04.453711 12187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:04.453755 12187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:04.453815 12187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:04.453851 12187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:04.463675 12187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:04.465119 12187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:04.466531 12187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:04.478639 12193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:04.478545 12195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:04.478612 12199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:04.479429 12187 server_base.cc:1034] running on GCE node
I20260811 00:18:04.480232 12187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:04.481714 12187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:04.482939 12187 hybrid_clock.cc:648] HybridClock initialized: now 1786407484482893 us; error 56 us; skew 500 ppm
I20260811 00:18:04.483281 12187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:04 dist-test-slave-8hhm krb5kdc[12164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407484, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:04.486593 12187 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:18:04.488086 12187 webserver.cc:460] Webserver started at http://127.0.0.1:33389/ using document root <none> and password file <none>
I20260811 00:18:04.488795 12187 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:04.488946 12187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:04.489321 12187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:04.492569 12187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "2bbaa72c32c34233a0f89aecd518ef2f"
format_stamp: "Formatted at 2026-08-11 00:18:04 on dist-test-slave-8hhm"
server_key: "41d0a2b5c720ac8b15daee6e6afb87bc"
server_key_iv: "107a91e7a8038195b6d5af7c4414e6b3"
server_key_version: "encryptionkey@0"
I20260811 00:18:04.493469 12187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "2bbaa72c32c34233a0f89aecd518ef2f"
format_stamp: "Formatted at 2026-08-11 00:18:04 on dist-test-slave-8hhm"
server_key: "41d0a2b5c720ac8b15daee6e6afb87bc"
server_key_iv: "107a91e7a8038195b6d5af7c4414e6b3"
server_key_version: "encryptionkey@0"
I20260811 00:18:04.498721 12187 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 00:18:04.502451 12210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:04.503777 12187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:18:04.503979 12187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "2bbaa72c32c34233a0f89aecd518ef2f"
format_stamp: "Formatted at 2026-08-11 00:18:04 on dist-test-slave-8hhm"
server_key: "41d0a2b5c720ac8b15daee6e6afb87bc"
server_key_iv: "107a91e7a8038195b6d5af7c4414e6b3"
server_key_version: "encryptionkey@0"
I20260811 00:18:04.504204 12187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:04.518507 12187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:04.523051 12187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:04.523383 12187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:04.542495 12187 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39965
I20260811 00:18:04.542542 12295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39965 every 8 connection(s)
I20260811 00:18:04.544030 12187 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:18:04.545933 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 12187
I20260811 00:18:04.546141 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:18:04.546598 7544 external_mini_cluster.cc:1428] Setting key 6bfa889fed0a86a13ff0c44440d1ad96
I20260811 00:18:04.548306 12296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:04.560138 12301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:04.548472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:59865 (local address 127.0.0.1:39965)
0811 00:18:04.549292 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:04.549306 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:04.549354 (+ 48us) server_negotiation.cc:407] Connection header received
0811 00:18:04.550875 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:04.550895 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:04.551269 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:04.551583 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:04.552236 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:04.553647 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:04.555574 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:04.556236 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:04.556378 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:04.557811 (+ 1433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:04.557818 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:04.557831 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:18:04.558394 (+ 563us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:04.558469 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:18:04.558491 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:04.558636 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:04.558976 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 00:18:04.559180 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":354,"threads_started":1}
I20260811 00:18:04.561340 12296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f: Bootstrap starting.
I20260811 00:18:04.563874 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:18:04.564649 12296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:04.565963 12296 log.cc:824] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:04.569538 12296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f: No bootstrap required, opened a new log
I20260811 00:18:04.576300 12301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:04.569028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:34619 (local address 127.0.0.1:39965)
0811 00:18:04.569317 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:04.569326 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:04.569842 (+ 516us) server_negotiation.cc:407] Connection header received
0811 00:18:04.570044 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:04.570048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:04.570111 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:04.570194 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:04.570826 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:04.571571 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:04.573578 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:04.574046 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:04.574138 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:04.575476 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:04.575480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:04.575484 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:18:04.575504 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:04.575541 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:18:04.575559 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:04.575637 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:04.575851 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 00:18:04.575901 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 00:18:04.579249 12296 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbaa72c32c34233a0f89aecd518ef2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39965 } }
I20260811 00:18:04.579667 12296 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:18:04.579785 12296 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bbaa72c32c34233a0f89aecd518ef2f, State: Initialized, Role: FOLLOWER
I20260811 00:18:04.580374 12296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbaa72c32c34233a0f89aecd518ef2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39965 } }
I20260811 00:18:04.580626 12296 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:18:04.580777 12296 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:18:04.580947 12296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:04.583222 12296 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbaa72c32c34233a0f89aecd518ef2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39965 } }
I20260811 00:18:04.583791 12296 leader_election.cc:302] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bbaa72c32c34233a0f89aecd518ef2f; no voters:
I20260811 00:18:04.584403 12296 leader_election.cc:288] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:18:04.584549 12316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:18:04.585265 12316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f [term 1 LEADER]: Becoming Leader. State: Replica: 2bbaa72c32c34233a0f89aecd518ef2f, State: Running, Role: LEADER
I20260811 00:18:04.585830 12316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbaa72c32c34233a0f89aecd518ef2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39965 } }
I20260811 00:18:04.586663 12296 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:18:04.588549 12318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bbaa72c32c34233a0f89aecd518ef2f. Latest consensus state: current_term: 1 leader_uuid: "2bbaa72c32c34233a0f89aecd518ef2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbaa72c32c34233a0f89aecd518ef2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39965 } } }
I20260811 00:18:04.588924 12318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:04.590565 12322 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:18:04.590731 12317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2bbaa72c32c34233a0f89aecd518ef2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbaa72c32c34233a0f89aecd518ef2f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39965 } } }
I20260811 00:18:04.591107 12317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:04.596033 12322 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:18:04.605104 12322 catalog_manager.cc:1347] Generated new cluster ID: 76bf7322098445f8983f07f25b4c0917
I20260811 00:18:04.605259 12322 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:18:04.616816 12322 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:18:04.618256 12322 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:18:04.629359 12322 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2bbaa72c32c34233a0f89aecd518ef2f: Generated new TSK 0
I20260811 00:18:04.630271 12322 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:18:04.637816 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 12187
W20260811 00:18:04.640147 12289 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.202:59865
W20260811 00:18:04.640928 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 12187: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:18:04.641084 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 12187
W20260811 00:18:04.668184 12157 connection.cc:535] client connection to 127.0.0.1:39965 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39965): Connection reset by peer (error 104)
2026-08-11T00:18:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2254 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:18:04 dist-test-slave-8hhm krb5kdc[12342](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:18:04 dist-test-slave-8hhm krb5kdc[12342](info): set up 2 sockets
Aug 11 00:18:04 dist-test-slave-8hhm krb5kdc[12342](info): commencing operation
krb5kdc: starting...
W20260811 00:18:06.016709 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.329s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:18:06 dist-test-slave-8hhm krb5kdc[12342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:18:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:18:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:18:06.360628 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38389
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:06.701370 12361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:06.701644 12361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:06.701808 12361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:06.708978 12361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:06.709091 12361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:06.709138 12361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:06.709290 12361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:06.709353 12361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:06.719058 12361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:06.720356 12361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:06.721724 12361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:06.733719 12371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:06.733915 12373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:06.734210 12370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:06.734719 12361 server_base.cc:1034] running on GCE node
I20260811 00:18:06.735441 12361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:06.736711 12361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:06.737907 12361 hybrid_clock.cc:648] HybridClock initialized: now 1786407486737845 us; error 50 us; skew 500 ppm
I20260811 00:18:06.738209 12361 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:06 dist-test-slave-8hhm krb5kdc[12342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407486, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:06.741418 12361 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:18:06.742995 12361 webserver.cc:460] Webserver started at http://127.0.0.1:43457/ using document root <none> and password file <none>
I20260811 00:18:06.743691 12361 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:06.743829 12361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:06.744133 12361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:06.747126 12361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "cb439388e3bf445eb744a3fcdd93bde0"
format_stamp: "Formatted at 2026-08-11 00:18:06 on dist-test-slave-8hhm"
server_key: "372e0e755f88c23e202367542a92be6d"
server_key_iv: "46c78782653ae405d840e2bba1e5f88f"
server_key_version: "encryptionkey@0"
I20260811 00:18:06.747900 12361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "cb439388e3bf445eb744a3fcdd93bde0"
format_stamp: "Formatted at 2026-08-11 00:18:06 on dist-test-slave-8hhm"
server_key: "372e0e755f88c23e202367542a92be6d"
server_key_iv: "46c78782653ae405d840e2bba1e5f88f"
server_key_version: "encryptionkey@0"
I20260811 00:18:06.752979 12361 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 00:18:06.757865 12383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:06.759794 12361 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 00:18:06.759997 12361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "cb439388e3bf445eb744a3fcdd93bde0"
format_stamp: "Formatted at 2026-08-11 00:18:06 on dist-test-slave-8hhm"
server_key: "372e0e755f88c23e202367542a92be6d"
server_key_iv: "46c78782653ae405d840e2bba1e5f88f"
server_key_version: "encryptionkey@0"
I20260811 00:18:06.760267 12361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:06.767396 12361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:06.771431 12361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:06.771754 12361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:06.791110 12361 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38389
I20260811 00:18:06.791162 12468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38389 every 8 connection(s)
I20260811 00:18:06.792596 12361 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:18:06.796165 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 12361
I20260811 00:18:06.796355 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:18:06.796940 12470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:06.796957 7544 external_mini_cluster.cc:1428] Setting key 1d04245f75a2e8140a094d7e00b89447
I20260811 00:18:06.804972 12477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:06.798756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:50461 (local address 127.0.0.1:38389)
0811 00:18:06.799539 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:06.799549 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:06.799591 (+ 42us) server_negotiation.cc:407] Connection header received
0811 00:18:06.801683 (+ 2092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:06.801706 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:06.802061 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:06.802388 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:06.803476 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:06.803485 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:06.803497 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:18:06.803513 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:06.803569 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:18:06.803590 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:06.803756 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:06.803856 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 00:18:06.804063 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":352,"threads_started":1}
I20260811 00:18:06.808723 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:18:06.810931 12470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0: Bootstrap starting.
I20260811 00:18:06.815201 12470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:06.816639 12470 log.cc:824] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:06.816792 12477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:06.814143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:48265 (local address 127.0.0.1:38389)
0811 00:18:06.814362 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:06.814368 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:06.814786 (+ 418us) server_negotiation.cc:407] Connection header received
0811 00:18:06.814923 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:06.814929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:06.814999 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:06.815129 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:06.816085 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:06.816091 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:06.816096 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:18:06.816114 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:06.816165 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:18:06.816186 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:06.816313 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:06.816375 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 00:18:06.816457 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 00:18:06.819594 12470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0: No bootstrap required, opened a new log
I20260811 00:18:06.826924 12470 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb439388e3bf445eb744a3fcdd93bde0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38389 } }
I20260811 00:18:06.827340 12470 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:18:06.827446 12470 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb439388e3bf445eb744a3fcdd93bde0, State: Initialized, Role: FOLLOWER
I20260811 00:18:06.828034 12470 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb439388e3bf445eb744a3fcdd93bde0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38389 } }
I20260811 00:18:06.828303 12470 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:18:06.828454 12470 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:18:06.828640 12470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:06.830914 12470 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb439388e3bf445eb744a3fcdd93bde0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38389 } }
I20260811 00:18:06.831408 12470 leader_election.cc:302] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb439388e3bf445eb744a3fcdd93bde0; no voters:
I20260811 00:18:06.832042 12470 leader_election.cc:288] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:18:06.832201 12491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:18:06.833366 12491 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0 [term 1 LEADER]: Becoming Leader. State: Replica: cb439388e3bf445eb744a3fcdd93bde0, State: Running, Role: LEADER
I20260811 00:18:06.833905 12491 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb439388e3bf445eb744a3fcdd93bde0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38389 } }
I20260811 00:18:06.834265 12470 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:18:06.836503 12495 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb439388e3bf445eb744a3fcdd93bde0. Latest consensus state: current_term: 1 leader_uuid: "cb439388e3bf445eb744a3fcdd93bde0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb439388e3bf445eb744a3fcdd93bde0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38389 } } }
I20260811 00:18:06.836846 12493 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb439388e3bf445eb744a3fcdd93bde0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb439388e3bf445eb744a3fcdd93bde0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38389 } } }
I20260811 00:18:06.837143 12493 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:06.837149 12495 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:06.840932 12501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:18:06.846506 12501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:18:06.855015 12501 catalog_manager.cc:1347] Generated new cluster ID: 74a2922cee344ba5ab216ed9e12c0837
I20260811 00:18:06.855123 12501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:18:06.865913 12501 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:18:06.866398 12501 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:18:06.872133 12501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb439388e3bf445eb744a3fcdd93bde0: Generated new TSK 0
I20260811 00:18:06.873109 12501 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:18:06.880334 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 12361
W20260811 00:18:06.881441 12462 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.202:50461
W20260811 00:18:06.882288 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 12361: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:18:06.882467 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 12361
2026-08-11T00:18:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2238 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:18:07 dist-test-slave-8hhm krb5kdc[12525](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:18:07 dist-test-slave-8hhm krb5kdc[12525](info): set up 2 sockets
Aug 11 00:18:07 dist-test-slave-8hhm krb5kdc[12525](info): commencing operation
krb5kdc: starting...
W20260811 00:18:08.243506 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.317s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:18:08 dist-test-slave-8hhm krb5kdc[12525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:18:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:18:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:18:08.546808 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33133
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:08.886369 12546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:08.886778 12546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:08.886966 12546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:08.894551 12546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:08.894647 12546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:08.894699 12546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:08.894781 12546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:08.894869 12546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:08.904156 12546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33133
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:08.905493 12546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:08.906766 12546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:08.918792 12553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:08.918762 12555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:08.918849 12557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:08.919500 12546 server_base.cc:1034] running on GCE node
I20260811 00:18:08.920289 12546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:08.921702 12546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:08.922936 12546 hybrid_clock.cc:648] HybridClock initialized: now 1786407488922905 us; error 40 us; skew 500 ppm
I20260811 00:18:08.923287 12546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:08 dist-test-slave-8hhm krb5kdc[12525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407488, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:08.930958 12546 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:18:08.932439 12546 webserver.cc:460] Webserver started at http://127.0.0.1:45373/ using document root <none> and password file <none>
I20260811 00:18:08.933174 12546 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:08.933367 12546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:08.933670 12546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:08.936645 12546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "7bcc68b5e0e148019e7b815e7ea34924"
format_stamp: "Formatted at 2026-08-11 00:18:08 on dist-test-slave-8hhm"
server_key: "efdf5fb1ea4f67fdcb6f5967fd72b90b"
server_key_iv: "9188f2d9c37d9f23cb8872797cd2a127"
server_key_version: "encryptionkey@0"
I20260811 00:18:08.937508 12546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "7bcc68b5e0e148019e7b815e7ea34924"
format_stamp: "Formatted at 2026-08-11 00:18:08 on dist-test-slave-8hhm"
server_key: "efdf5fb1ea4f67fdcb6f5967fd72b90b"
server_key_iv: "9188f2d9c37d9f23cb8872797cd2a127"
server_key_version: "encryptionkey@0"
I20260811 00:18:08.942617 12546 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 00:18:08.946171 12566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:08.947386 12546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 00:18:08.947568 12546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "7bcc68b5e0e148019e7b815e7ea34924"
format_stamp: "Formatted at 2026-08-11 00:18:08 on dist-test-slave-8hhm"
server_key: "efdf5fb1ea4f67fdcb6f5967fd72b90b"
server_key_iv: "9188f2d9c37d9f23cb8872797cd2a127"
server_key_version: "encryptionkey@0"
I20260811 00:18:08.947767 12546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:08.963972 12546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:08.968106 12546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:08.968420 12546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:08.986757 12546 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33133
I20260811 00:18:08.986793 12657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33133 every 8 connection(s)
I20260811 00:18:08.988215 12546 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:18:08.991940 12659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:08.996541 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 12546
I20260811 00:18:08.996755 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:18:08.997206 7544 external_mini_cluster.cc:1428] Setting key c5f5759bc0654dd7e145734dd7589321
I20260811 00:18:09.004134 12659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924: Bootstrap starting.
I20260811 00:18:09.007870 12659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:09.009037 12659 log.cc:824] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:09.012224 12659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924: No bootstrap required, opened a new log
Aug 11 00:18:09 dist-test-slave-8hhm krb5kdc[12525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 00:18:09.022528 12659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bcc68b5e0e148019e7b815e7ea34924" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33133 } }
I20260811 00:18:09.022966 12659 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:18:09.023080 12659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bcc68b5e0e148019e7b815e7ea34924, State: Initialized, Role: FOLLOWER
I20260811 00:18:09.023775 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:08.999090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:42805 (local address 127.0.0.1:33133)
0811 00:18:08.999915 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:08.999930 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:08.999984 (+ 54us) server_negotiation.cc:407] Connection header received
0811 00:18:09.002509 (+ 2525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:09.002539 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:09.003039 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:09.003491 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:09.004975 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:09.007128 (+ 2153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:09.008534 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:09.009636 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:09.009831 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:09.014870 (+ 5039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:09.014880 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:09.014894 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:09.014938 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:09.018201 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:09.019177 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:09.019190 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:09.019200 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:09.019399 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:09.019979 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:09.019984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:09.019987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:09.020724 (+ 737us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:09.021062 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:09.022073 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0811 00:18:09.022428 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":342,"threads_started":1}
I20260811 00:18:09.023744 12659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bcc68b5e0e148019e7b815e7ea34924" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33133 } }
I20260811 00:18:09.024052 12659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:18:09.024219 12659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:18:09.024420 12659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:09.027335 12659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bcc68b5e0e148019e7b815e7ea34924" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33133 } }
I20260811 00:18:09.027864 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:18:09.027834 12659 leader_election.cc:302] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7bcc68b5e0e148019e7b815e7ea34924; no voters:
I20260811 00:18:09.028467 12659 leader_election.cc:288] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:18:09.028581 12665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:18:09.029945 12665 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924 [term 1 LEADER]: Becoming Leader. State: Replica: 7bcc68b5e0e148019e7b815e7ea34924, State: Running, Role: LEADER
I20260811 00:18:09.030423 12665 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bcc68b5e0e148019e7b815e7ea34924" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33133 } }
I20260811 00:18:09.030568 12659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:18:09.033646 12666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7bcc68b5e0e148019e7b815e7ea34924" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bcc68b5e0e148019e7b815e7ea34924" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33133 } } }
I20260811 00:18:09.033520 12667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bcc68b5e0e148019e7b815e7ea34924. Latest consensus state: current_term: 1 leader_uuid: "7bcc68b5e0e148019e7b815e7ea34924" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bcc68b5e0e148019e7b815e7ea34924" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33133 } } }
I20260811 00:18:09.033942 12666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:09.033942 12667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:09.034778 12678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:18:09.048539 12678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:18:09.060969 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:09.041614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:44663 (local address 127.0.0.1:33133)
0811 00:18:09.041780 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:09.041785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:09.042605 (+ 820us) server_negotiation.cc:407] Connection header received
0811 00:18:09.042927 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:09.042933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:09.043024 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:09.043137 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:09.045205 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:09.046504 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:09.048398 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:09.049156 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:09.049314 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:09.053113 (+ 3799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:09.053121 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:09.053126 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:09.053164 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:09.057085 (+ 3921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:09.058197 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:09.058203 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:09.058206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:09.058367 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:09.059001 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:09.059007 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:09.059011 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:09.059422 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:09.059668 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:09.060373 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 00:18:09.060598 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":57}
I20260811 00:18:09.072427 12678 catalog_manager.cc:1347] Generated new cluster ID: 23df68fa9137465f8ae6aff577a76e4d
I20260811 00:18:09.072556 12678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:18:09.096987 12678 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:18:09.098271 12678 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:18:09.112947 12678 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7bcc68b5e0e148019e7b815e7ea34924: Generated new TSK 0
I20260811 00:18:09.113880 12678 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:18:09.126984 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 12546
I20260811 00:18:09.128177 12650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:18:09.220208 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 12546
W20260811 00:18:09.246050 12521 connection.cc:535] client connection to 127.0.0.1:33133 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33133): Connection reset by peer (error 104)
2026-08-11T00:18:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:18:09 dist-test-slave-8hhm krb5kdc[12706](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:18:09 dist-test-slave-8hhm krb5kdc[12706](info): set up 2 sockets
Aug 11 00:18:09 dist-test-slave-8hhm krb5kdc[12706](info): commencing operation
krb5kdc: starting...
W20260811 00:18:10.575537 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.003s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:18:10 dist-test-slave-8hhm krb5kdc[12706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:18:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:18:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:18:10.884614 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34515
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:11.222185 12725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:11.222505 12725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:11.222681 12725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:11.229734 12725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:11.229868 12725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:11.229980 12725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:11.230080 12725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:11.230161 12725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:11.239782 12725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34515
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:11.241065 12725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:11.242391 12725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:11.254520 12733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:11.254487 12738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:11.254700 12735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:11.255493 12725 server_base.cc:1034] running on GCE node
I20260811 00:18:11.256333 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:11.257694 12725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:11.258889 12725 hybrid_clock.cc:648] HybridClock initialized: now 1786407491258852 us; error 28 us; skew 500 ppm
I20260811 00:18:11.259243 12725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:11 dist-test-slave-8hhm krb5kdc[12706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407491, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:11.262476 12725 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:18:11.264015 12725 webserver.cc:460] Webserver started at http://127.0.0.1:39569/ using document root <none> and password file <none>
I20260811 00:18:11.264673 12725 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:11.264822 12725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:11.265126 12725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:11.268096 12725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "21b5f55a5e904527a8e40188a4bc66f8"
format_stamp: "Formatted at 2026-08-11 00:18:11 on dist-test-slave-8hhm"
server_key: "91598a6b6719351ef494bce0b36199fc"
server_key_iv: "abe31c26e9e9d82fb7b2437cfb2f183a"
server_key_version: "encryptionkey@0"
I20260811 00:18:11.268875 12725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "21b5f55a5e904527a8e40188a4bc66f8"
format_stamp: "Formatted at 2026-08-11 00:18:11 on dist-test-slave-8hhm"
server_key: "91598a6b6719351ef494bce0b36199fc"
server_key_iv: "abe31c26e9e9d82fb7b2437cfb2f183a"
server_key_version: "encryptionkey@0"
I20260811 00:18:11.274062 12725 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 00:18:11.277350 12755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:11.278573 12725 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:18:11.278751 12725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "21b5f55a5e904527a8e40188a4bc66f8"
format_stamp: "Formatted at 2026-08-11 00:18:11 on dist-test-slave-8hhm"
server_key: "91598a6b6719351ef494bce0b36199fc"
server_key_iv: "abe31c26e9e9d82fb7b2437cfb2f183a"
server_key_version: "encryptionkey@0"
I20260811 00:18:11.278944 12725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:11.315968 12725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:11.319962 12725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:11.320261 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:11.338928 12725 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34515
I20260811 00:18:11.338964 12828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34515 every 8 connection(s)
I20260811 00:18:11.340283 12725 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:18:11.343870 12829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:11.345124 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 12725
I20260811 00:18:11.345381 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:18:11.345870 7544 external_mini_cluster.cc:1428] Setting key bb73a0414d331f34debe96ca994bb3d6
I20260811 00:18:11.358644 12829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8: Bootstrap starting.
I20260811 00:18:11.360489 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:11.347660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:46779 (local address 127.0.0.1:34515)
0811 00:18:11.348428 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:11.348453 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:11.348502 (+ 49us) server_negotiation.cc:407] Connection header received
0811 00:18:11.350708 (+ 2206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:11.350736 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:11.351261 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:11.351725 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:11.352215 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:11.354083 (+ 1868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:11.355392 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:11.356488 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:11.356671 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:11.357809 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:11.357821 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:11.357839 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:18:11.358518 (+ 679us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:11.358621 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:18:11.358667 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:11.358820 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:11.359083 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 00:18:11.359238 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":318,"threads_started":1}
I20260811 00:18:11.362272 12829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:11.363505 12829 log.cc:824] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:11.364198 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:18:11.366184 12829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8: No bootstrap required, opened a new log
I20260811 00:18:11.375109 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:11.368674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:57175 (local address 127.0.0.1:34515)
0811 00:18:11.368861 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:11.368867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:11.369171 (+ 304us) server_negotiation.cc:407] Connection header received
0811 00:18:11.369397 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:11.369402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:11.369483 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:11.369586 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:11.370068 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:11.371223 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:11.372493 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:11.373205 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:11.373349 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:11.374418 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:11.374424 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:11.374428 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:18:11.374451 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:11.374491 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:18:11.374509 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:11.374627 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:11.374775 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 00:18:11.374820 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 00:18:11.375435 12829 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21b5f55a5e904527a8e40188a4bc66f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34515 } }
I20260811 00:18:11.375793 12829 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:18:11.375927 12829 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21b5f55a5e904527a8e40188a4bc66f8, State: Initialized, Role: FOLLOWER
I20260811 00:18:11.376483 12829 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21b5f55a5e904527a8e40188a4bc66f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34515 } }
I20260811 00:18:11.376709 12829 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:18:11.376863 12829 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:18:11.377027 12829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:11.379271 12829 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21b5f55a5e904527a8e40188a4bc66f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34515 } }
I20260811 00:18:11.379741 12829 leader_election.cc:302] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21b5f55a5e904527a8e40188a4bc66f8; no voters:
I20260811 00:18:11.380298 12829 leader_election.cc:288] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:18:11.380455 12851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:18:11.381599 12851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8 [term 1 LEADER]: Becoming Leader. State: Replica: 21b5f55a5e904527a8e40188a4bc66f8, State: Running, Role: LEADER
I20260811 00:18:11.382256 12851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21b5f55a5e904527a8e40188a4bc66f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34515 } }
I20260811 00:18:11.382534 12829 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:18:11.384945 12852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21b5f55a5e904527a8e40188a4bc66f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21b5f55a5e904527a8e40188a4bc66f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34515 } } }
I20260811 00:18:11.385326 12852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:11.385885 12854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21b5f55a5e904527a8e40188a4bc66f8. Latest consensus state: current_term: 1 leader_uuid: "21b5f55a5e904527a8e40188a4bc66f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21b5f55a5e904527a8e40188a4bc66f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34515 } } }
I20260811 00:18:11.386248 12854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:11.389801 12867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:18:11.394690 12867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:18:11.405176 12867 catalog_manager.cc:1347] Generated new cluster ID: 137ed6e6901a4d1abf4f5881a5fd6dac
I20260811 00:18:11.405309 12867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:18:11.418102 12867 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:18:11.419337 12867 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:18:11.441179 12867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21b5f55a5e904527a8e40188a4bc66f8: Generated new TSK 0
I20260811 00:18:11.442039 12867 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:18:11.451752 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 12725
W20260811 00:18:11.453604 12823 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.202:46779
W20260811 00:18:11.454321 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 12725: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:18:11.454464 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 12725
W20260811 00:18:11.483839 12703 connection.cc:535] client connection to 127.0.0.1:34515 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34515): Connection reset by peer (error 104)
2026-08-11T00:18:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2238 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:18:11 dist-test-slave-8hhm krb5kdc[12886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:18:11 dist-test-slave-8hhm krb5kdc[12886](info): set up 2 sockets
Aug 11 00:18:11 dist-test-slave-8hhm krb5kdc[12886](info): commencing operation
krb5kdc: starting...
W20260811 00:18:12.853144 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.350s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:18:13 dist-test-slave-8hhm krb5kdc[12886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:18:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:18:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 00:18:13.167868 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39135
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:13.500587 12909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:13.500862 12909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:13.501021 12909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:13.507773 12909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:13.507859 12909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:13.507988 12909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:13.508044 12909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:13.508080 12909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:13.517378 12909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39135
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:13.518579 12909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:13.519850 12909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:13.531387 12920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:13.531457 12918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:13.531435 12924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:13.532318 12909 server_base.cc:1034] running on GCE node
I20260811 00:18:13.533105 12909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:13.534446 12909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:13.535668 12909 hybrid_clock.cc:648] HybridClock initialized: now 1786407493535630 us; error 51 us; skew 500 ppm
I20260811 00:18:13.536031 12909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:13 dist-test-slave-8hhm krb5kdc[12886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407493, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:13.539077 12909 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 00:18:13.540405 12909 webserver.cc:460] Webserver started at http://127.0.0.1:33459/ using document root <none> and password file <none>
I20260811 00:18:13.541049 12909 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:13.541200 12909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:13.541545 12909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:13.544389 12909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "5fe1c72e37ed4a68a05f839472025cff"
format_stamp: "Formatted at 2026-08-11 00:18:13 on dist-test-slave-8hhm"
server_key: "1bdb086b1181b92c17ad88c7110d843b"
server_key_iv: "bd3ce9499911515fbce085f9dfc75cf2"
server_key_version: "encryptionkey@0"
I20260811 00:18:13.545264 12909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "5fe1c72e37ed4a68a05f839472025cff"
format_stamp: "Formatted at 2026-08-11 00:18:13 on dist-test-slave-8hhm"
server_key: "1bdb086b1181b92c17ad88c7110d843b"
server_key_iv: "bd3ce9499911515fbce085f9dfc75cf2"
server_key_version: "encryptionkey@0"
I20260811 00:18:13.550366 12909 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 00:18:13.553591 12935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:13.554765 12909 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260811 00:18:13.554941 12909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "5fe1c72e37ed4a68a05f839472025cff"
format_stamp: "Formatted at 2026-08-11 00:18:13 on dist-test-slave-8hhm"
server_key: "1bdb086b1181b92c17ad88c7110d843b"
server_key_iv: "bd3ce9499911515fbce085f9dfc75cf2"
server_key_version: "encryptionkey@0"
I20260811 00:18:13.555137 12909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:13.560563 12909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:13.564175 12909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:13.564451 12909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:13.582635 12909 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39135
I20260811 00:18:13.582708 13025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39135 every 8 connection(s)
I20260811 00:18:13.583935 12909 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:18:13.585963 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 12909
I20260811 00:18:13.586151 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:18:13.586621 7544 external_mini_cluster.cc:1428] Setting key 31f122413bab93063d87a2ed3b27ae11
I20260811 00:18:13.587834 13026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:13.594893 13032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:13.588425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:39871 (local address 127.0.0.1:39135)
0811 00:18:13.589279 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:13.589295 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:13.589346 (+ 51us) server_negotiation.cc:407] Connection header received
0811 00:18:13.591119 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:13.591144 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:13.591538 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:13.591986 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:13.592971 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:13.592979 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:13.592993 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:18:13.593011 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:13.593128 (+ 117us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:18:13.593158 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:13.593405 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:13.593536 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 00:18:13.593733 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":326,"threads_started":1}
I20260811 00:18:13.598264 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:18:13.602017 13026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff: Bootstrap starting.
I20260811 00:18:13.605898 13026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:13.605978 13032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:13.603451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.202:56051 (local address 127.0.0.1:39135)
0811 00:18:13.603658 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:13.603666 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:13.604077 (+ 411us) server_negotiation.cc:407] Connection header received
0811 00:18:13.604236 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:13.604239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:13.604278 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:13.604348 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:13.605278 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:13.605282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:13.605285 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:18:13.605298 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:13.605333 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:18:13.605346 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:13.605427 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:13.605576 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 00:18:13.605626 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 00:18:13.606993 13026 log.cc:824] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:13.609479 13026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff: No bootstrap required, opened a new log
I20260811 00:18:13.616221 13026 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fe1c72e37ed4a68a05f839472025cff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39135 } }
I20260811 00:18:13.616575 13026 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:18:13.616685 13026 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fe1c72e37ed4a68a05f839472025cff, State: Initialized, Role: FOLLOWER
I20260811 00:18:13.617197 13026 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fe1c72e37ed4a68a05f839472025cff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39135 } }
I20260811 00:18:13.617496 13026 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:18:13.617637 13026 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:18:13.617830 13026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:13.619864 13026 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fe1c72e37ed4a68a05f839472025cff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39135 } }
I20260811 00:18:13.620333 13026 leader_election.cc:302] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fe1c72e37ed4a68a05f839472025cff; no voters:
I20260811 00:18:13.620882 13026 leader_election.cc:288] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:18:13.621018 13043 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:18:13.621695 13043 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff [term 1 LEADER]: Becoming Leader. State: Replica: 5fe1c72e37ed4a68a05f839472025cff, State: Running, Role: LEADER
I20260811 00:18:13.622263 13043 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fe1c72e37ed4a68a05f839472025cff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39135 } }
I20260811 00:18:13.622884 13026 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:18:13.625411 13044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5fe1c72e37ed4a68a05f839472025cff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fe1c72e37ed4a68a05f839472025cff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39135 } } }
I20260811 00:18:13.625428 13045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fe1c72e37ed4a68a05f839472025cff. Latest consensus state: current_term: 1 leader_uuid: "5fe1c72e37ed4a68a05f839472025cff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fe1c72e37ed4a68a05f839472025cff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39135 } } }
I20260811 00:18:13.625911 13044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:13.625957 13045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:13.629894 13055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:18:13.635000 13055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:18:13.643788 13055 catalog_manager.cc:1347] Generated new cluster ID: 66680a381e564d04b2780c04ac12b65e
I20260811 00:18:13.643901 13055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:18:13.666702 13055 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:18:13.667176 13055 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:18:13.681079 13055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fe1c72e37ed4a68a05f839472025cff: Generated new TSK 0
I20260811 00:18:13.681859 13055 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:18:13.693244 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 12909
W20260811 00:18:13.694336 13018 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.202:39871
W20260811 00:18:13.694981 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 12909: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:18:13.695155 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 12909
2026-08-11T00:18:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2238 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:18:13 dist-test-slave-8hhm krb5kdc[13073](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:18:13 dist-test-slave-8hhm krb5kdc[13073](info): set up 2 sockets
Aug 11 00:18:13 dist-test-slave-8hhm krb5kdc[13073](info): commencing operation
krb5kdc: starting...
W20260811 00:18:15.071069 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.330s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:18:15 dist-test-slave-8hhm krb5kdc[13073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:18:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:18:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:18:15.392800 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:37757
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:34865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:37757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:15.744570 13094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:15.744859 13094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:15.745070 13094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:15.752305 13094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:15.752410 13094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:15.752532 13094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:15.752650 13094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:15.752717 13094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:15.762573 13094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:34865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:37757
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:37757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:15.763872 13094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:15.765203 13094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:15.777312 13104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:15.777277 13109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:15.777551 13106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:15.778247 13094 server_base.cc:1034] running on GCE node
I20260811 00:18:15.778975 13094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:15.780256 13094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:15.781451 13094 hybrid_clock.cc:648] HybridClock initialized: now 1786407495781391 us; error 58 us; skew 500 ppm
I20260811 00:18:15.781831 13094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:15 dist-test-slave-8hhm krb5kdc[13073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407495, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:15.791627 13094 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:18:15.793136 13094 webserver.cc:460] Webserver started at http://127.7.94.62:45869/ using document root <none> and password file <none>
I20260811 00:18:15.793891 13094 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:15.794000 13094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:15.794265 13094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:15.797348 13094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "6e4df023626949dc8ebbea948b16c49d"
format_stamp: "Formatted at 2026-08-11 00:18:15 on dist-test-slave-8hhm"
server_key: "a61730ed4fddb9bcb337ff2ab7d09e97"
server_key_iv: "a500e6e86ef3667d774554aff65ec022"
server_key_version: "encryptionkey@0"
I20260811 00:18:15.798074 13094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "6e4df023626949dc8ebbea948b16c49d"
format_stamp: "Formatted at 2026-08-11 00:18:15 on dist-test-slave-8hhm"
server_key: "a61730ed4fddb9bcb337ff2ab7d09e97"
server_key_iv: "a500e6e86ef3667d774554aff65ec022"
server_key_version: "encryptionkey@0"
I20260811 00:18:15.803094 13094 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 00:18:15.806690 13123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:15.808036 13094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 00:18:15.808224 13094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "6e4df023626949dc8ebbea948b16c49d"
format_stamp: "Formatted at 2026-08-11 00:18:15 on dist-test-slave-8hhm"
server_key: "a61730ed4fddb9bcb337ff2ab7d09e97"
server_key_iv: "a500e6e86ef3667d774554aff65ec022"
server_key_version: "encryptionkey@0"
I20260811 00:18:15.808421 13094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:15.830921 13094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:15.835069 13094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:15.835371 13094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:15.854369 13094 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:37757
I20260811 00:18:15.854460 13204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:37757 every 8 connection(s)
I20260811 00:18:15.855782 13094 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:18:15.859418 13206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:15.865836 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 13094
I20260811 00:18:15.866029 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:18:15.866506 7544 external_mini_cluster.cc:1428] Setting key 8c3d1ac765f79396991dd5009dfab4bd
I20260811 00:18:15.869896 13206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d: Bootstrap starting.
I20260811 00:18:15.873502 13206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:15.874673 13206 log.cc:824] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:15.877627 13206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d: No bootstrap required, opened a new log
Aug 11 00:18:15 dist-test-slave-8hhm krb5kdc[13073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:18:15.886047 13206 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4df023626949dc8ebbea948b16c49d" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 37757 } }
I20260811 00:18:15.886447 13206 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:18:15.886552 13206 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e4df023626949dc8ebbea948b16c49d, State: Initialized, Role: FOLLOWER
I20260811 00:18:15.887203 13206 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4df023626949dc8ebbea948b16c49d" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 37757 } }
I20260811 00:18:15.887471 13206 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:18:15.887658 13206 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:18:15.887869 13206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:15.889955 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:15.868279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46601 (local address 127.7.94.62:37757)
0811 00:18:15.869085 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:15.869096 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:15.869137 (+ 41us) server_negotiation.cc:407] Connection header received
0811 00:18:15.870927 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:15.870949 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:15.871326 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:15.871630 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:15.873585 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:15.874974 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:15.876671 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:15.877388 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:15.877508 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:15.882272 (+ 4764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:15.882282 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:15.882296 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:15.882331 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:15.885512 (+ 3181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:15.886278 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:15.886284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:15.886294 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:15.886511 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:15.887030 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:15.887035 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:15.887039 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:15.887678 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:15.888009 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:15.888541 (+ 532us) server_negotiation.cc:299] Negotiation successful
0811 00:18:15.888885 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":388,"threads_started":1}
I20260811 00:18:15.890517 13206 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4df023626949dc8ebbea948b16c49d" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 37757 } }
I20260811 00:18:15.891059 13206 leader_election.cc:302] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e4df023626949dc8ebbea948b16c49d; no voters:
I20260811 00:18:15.891691 13206 leader_election.cc:288] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:18:15.891850 13213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:18:15.892513 13213 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d [term 1 LEADER]: Becoming Leader. State: Replica: 6e4df023626949dc8ebbea948b16c49d, State: Running, Role: LEADER
I20260811 00:18:15.893162 13213 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4df023626949dc8ebbea948b16c49d" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 37757 } }
I20260811 00:18:15.894614 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:18:15.895035 13206 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:18:15.902935 13216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e4df023626949dc8ebbea948b16c49d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4df023626949dc8ebbea948b16c49d" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 37757 } } }
I20260811 00:18:15.903383 13216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:15.904074 13215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e4df023626949dc8ebbea948b16c49d. Latest consensus state: current_term: 1 leader_uuid: "6e4df023626949dc8ebbea948b16c49d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4df023626949dc8ebbea948b16c49d" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 37757 } } }
I20260811 00:18:15.904386 13215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:15.904297 13235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:18:15.914508 13235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:18:15.926870 13235 catalog_manager.cc:1347] Generated new cluster ID: a8255fb40fb04c00aaad5a57cdd004ba
I20260811 00:18:15.926981 13235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:18:15.936689 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:15.921766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34177 (local address 127.7.94.62:37757)
0811 00:18:15.921948 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:15.921957 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:15.922563 (+ 606us) server_negotiation.cc:407] Connection header received
0811 00:18:15.922789 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:15.922795 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:15.922876 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:15.923007 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:15.924442 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:15.925682 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:15.927462 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:15.927909 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:15.928039 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:15.930670 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:15.930675 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:15.930678 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:15.930708 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:15.933745 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:15.934512 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:15.934519 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:15.934530 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:15.934668 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:15.935108 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:15.935115 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:15.935122 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:15.935441 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:15.935730 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:15.936125 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 00:18:15.936299 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 00:18:15.961817 13235 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:18:15.963678 13235 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:18:15.972312 13235 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e4df023626949dc8ebbea948b16c49d: Generated new TSK 0
I20260811 00:18:15.973361 13235 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:18:15.985198 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 13094
I20260811 00:18:15.987025 13199 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:18:16.081511 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 13094
W20260811 00:18:16.107909 13071 connection.cc:535] client connection to 127.7.94.62:37757 recv error: Network error: failed to read from TLS socket (remote: 127.7.94.62:37757): Connection reset by peer (error 104)
2026-08-11T00:18:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2385 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:18:16 dist-test-slave-8hhm krb5kdc[13268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:18:16 dist-test-slave-8hhm krb5kdc[13268](info): set up 2 sockets
Aug 11 00:18:16 dist-test-slave-8hhm krb5kdc[13268](info): commencing operation
krb5kdc: starting...
W20260811 00:18:17.467074 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.340s user 0.004s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:18:17 dist-test-slave-8hhm krb5kdc[13268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:18:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:18:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:18:17.794658 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:36611
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:45399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:36611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:18.163134 13290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:18.163419 13290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:18.163508 13290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:18.170671 13290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:18.170807 13290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:18.170913 13290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:18.171015 13290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:18.171096 13290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:18.180766 13290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:45399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:36611
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:36611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:18.182089 13290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:18.183442 13290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:18.195941 13297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:18.196040 13298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:18.196004 13300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:18.196974 13290 server_base.cc:1034] running on GCE node
I20260811 00:18:18.197816 13290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:18.199246 13290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:18.200457 13290 hybrid_clock.cc:648] HybridClock initialized: now 1786407498200423 us; error 28 us; skew 500 ppm
I20260811 00:18:18.200824 13290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:18 dist-test-slave-8hhm krb5kdc[13268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407498, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:18.204051 13290 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:18:18.205575 13290 webserver.cc:460] Webserver started at http://127.7.94.62:37209/ using document root <none> and password file <none>
I20260811 00:18:18.206333 13290 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:18.206491 13290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:18.206802 13290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:18.209863 13290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "a23087b1a67e467c94ea98dabfb93375"
format_stamp: "Formatted at 2026-08-11 00:18:18 on dist-test-slave-8hhm"
server_key: "80d1283feaff151c1679a887199562c3"
server_key_iv: "b721e9292068dd9a342f997cfa2fef77"
server_key_version: "encryptionkey@0"
I20260811 00:18:18.210618 13290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "a23087b1a67e467c94ea98dabfb93375"
format_stamp: "Formatted at 2026-08-11 00:18:18 on dist-test-slave-8hhm"
server_key: "80d1283feaff151c1679a887199562c3"
server_key_iv: "b721e9292068dd9a342f997cfa2fef77"
server_key_version: "encryptionkey@0"
I20260811 00:18:18.215869 13290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:18:18.219295 13310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:18.220635 13290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:18:18.220820 13290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "a23087b1a67e467c94ea98dabfb93375"
format_stamp: "Formatted at 2026-08-11 00:18:18 on dist-test-slave-8hhm"
server_key: "80d1283feaff151c1679a887199562c3"
server_key_iv: "b721e9292068dd9a342f997cfa2fef77"
server_key_version: "encryptionkey@0"
I20260811 00:18:18.221028 13290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:18.252009 13290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:18.256001 13290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:18.256299 13290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:18.275317 13290 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:36611
I20260811 00:18:18.275354 13390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:36611 every 8 connection(s)
I20260811 00:18:18.276705 13290 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:18:18.278867 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 13290
I20260811 00:18:18.279072 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:18:18.279698 7544 external_mini_cluster.cc:1428] Setting key aafb0215c0d53f363c5382ad33bf48e9
I20260811 00:18:18.280963 13392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:18.292409 13398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:18.281627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44453 (local address 127.7.94.62:36611)
0811 00:18:18.282387 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:18.282398 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:18.282438 (+ 40us) server_negotiation.cc:407] Connection header received
0811 00:18:18.284006 (+ 1568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:18.284026 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:18.284407 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:18.284814 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:18.285390 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:18.286716 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:18.288050 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:18.288676 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:18.288869 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:18.290247 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:18.290254 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:18.290266 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:18:18.290792 (+ 526us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:18.290868 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:18:18.290894 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:18.291076 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:18.291359 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 00:18:18.291503 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":334,"threads_started":1}
I20260811 00:18:18.292902 13392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375: Bootstrap starting.
I20260811 00:18:18.296136 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:18:18.296172 13392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:18.297127 13392 log.cc:824] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:18.299950 13392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375: No bootstrap required, opened a new log
I20260811 00:18:18.307770 13398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:18.300896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36681 (local address 127.7.94.62:36611)
0811 00:18:18.301095 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:18.301101 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:18.301498 (+ 397us) server_negotiation.cc:407] Connection header received
0811 00:18:18.301684 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:18.301688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:18.301743 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:18.301823 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:18.302441 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:18.303216 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:18.305141 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:18.305681 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:18.305776 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:18.306990 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:18.306994 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:18.306997 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:18:18.307012 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:18.307065 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:18:18.307079 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:18.307177 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:18.307385 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 00:18:18.307423 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 00:18:18.309808 13392 raft_consensus.cc:348] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23087b1a67e467c94ea98dabfb93375" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36611 } }
I20260811 00:18:18.310184 13392 raft_consensus.cc:374] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:18:18.310292 13392 raft_consensus.cc:729] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a23087b1a67e467c94ea98dabfb93375, State: Initialized, Role: FOLLOWER
I20260811 00:18:18.310921 13392 consensus_queue.cc:260] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23087b1a67e467c94ea98dabfb93375" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36611 } }
I20260811 00:18:18.311182 13392 raft_consensus.cc:388] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:18:18.311347 13392 raft_consensus.cc:482] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:18:18.311525 13392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:18.313730 13392 raft_consensus.cc:504] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23087b1a67e467c94ea98dabfb93375" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36611 } }
I20260811 00:18:18.314225 13392 leader_election.cc:302] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a23087b1a67e467c94ea98dabfb93375; no voters:
I20260811 00:18:18.314793 13392 leader_election.cc:288] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:18:18.314924 13416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:18:18.316360 13416 raft_consensus.cc:686] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375 [term 1 LEADER]: Becoming Leader. State: Replica: a23087b1a67e467c94ea98dabfb93375, State: Running, Role: LEADER
I20260811 00:18:18.316910 13416 consensus_queue.cc:237] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23087b1a67e467c94ea98dabfb93375" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36611 } }
I20260811 00:18:18.317037 13392 sys_catalog.cc:564] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:18:18.319852 13421 sys_catalog.cc:455] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a23087b1a67e467c94ea98dabfb93375. Latest consensus state: current_term: 1 leader_uuid: "a23087b1a67e467c94ea98dabfb93375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23087b1a67e467c94ea98dabfb93375" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36611 } } }
I20260811 00:18:18.320292 13421 sys_catalog.cc:458] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:18.325176 13419 sys_catalog.cc:455] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a23087b1a67e467c94ea98dabfb93375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a23087b1a67e467c94ea98dabfb93375" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 36611 } } }
I20260811 00:18:18.325489 13428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:18:18.325508 13419 sys_catalog.cc:458] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:18.331880 13428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:18:18.340898 13428 catalog_manager.cc:1347] Generated new cluster ID: 2c6fc98fa9a443bf8da5e552e3cee5be
I20260811 00:18:18.341027 13428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:18:18.350199 13428 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:18:18.351877 13428 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:18:18.362011 13428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a23087b1a67e467c94ea98dabfb93375: Generated new TSK 0
I20260811 00:18:18.363049 13428 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:18:18.369015 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 13290
W20260811 00:18:18.370513 13381 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44453
W20260811 00:18:18.371259 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 13290: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:18:18.371407 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 13290
W20260811 00:18:18.401674 13263 connection.cc:535] client connection to 127.7.94.62:36611 recv error: Network error: failed to read from TLS socket (remote: 127.7.94.62:36611): Connection reset by peer (error 104)
2026-08-11T00:18:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2293 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:18:18 dist-test-slave-8hhm krb5kdc[13452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:18:18 dist-test-slave-8hhm krb5kdc[13452](info): set up 2 sockets
Aug 11 00:18:18 dist-test-slave-8hhm krb5kdc[13452](info): commencing operation
krb5kdc: starting...
W20260811 00:18:19.790915 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.370s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:18:20 dist-test-slave-8hhm krb5kdc[13452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:18:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:18:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:18:20.122460 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:43777
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:45425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:43777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:20.467744 13474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:20.468093 13474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:20.468289 13474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:20.475571 13474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:20.475721 13474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:20.475805 13474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:20.475898 13474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:20.476016 13474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:20.485576 13474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:45425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:43777
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:43777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:20.486940 13474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:20.488293 13474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:20.500206 13479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:20.500293 13480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:20.500167 13482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:20.501375 13474 server_base.cc:1034] running on GCE node
I20260811 00:18:20.502086 13474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:20.503420 13474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:20.504635 13474 hybrid_clock.cc:648] HybridClock initialized: now 1786407500504574 us; error 54 us; skew 500 ppm
I20260811 00:18:20.504940 13474 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:20 dist-test-slave-8hhm krb5kdc[13452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407500, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:20.508211 13474 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:18:20.509758 13474 webserver.cc:460] Webserver started at http://127.7.94.62:36377/ using document root <none> and password file <none>
I20260811 00:18:20.510391 13474 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:20.510502 13474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:20.510763 13474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:20.513813 13474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "369645504fdf4454b9666a61324d517a"
format_stamp: "Formatted at 2026-08-11 00:18:20 on dist-test-slave-8hhm"
server_key: "919b03209b683cfcd9c7533010e6330a"
server_key_iv: "8961e03b352f7b02a0cb89d777b61233"
server_key_version: "encryptionkey@0"
I20260811 00:18:20.514513 13474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "369645504fdf4454b9666a61324d517a"
format_stamp: "Formatted at 2026-08-11 00:18:20 on dist-test-slave-8hhm"
server_key: "919b03209b683cfcd9c7533010e6330a"
server_key_iv: "8961e03b352f7b02a0cb89d777b61233"
server_key_version: "encryptionkey@0"
I20260811 00:18:20.519569 13474 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:18:20.522918 13493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:20.524222 13474 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 00:18:20.524405 13474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "369645504fdf4454b9666a61324d517a"
format_stamp: "Formatted at 2026-08-11 00:18:20 on dist-test-slave-8hhm"
server_key: "919b03209b683cfcd9c7533010e6330a"
server_key_iv: "8961e03b352f7b02a0cb89d777b61233"
server_key_version: "encryptionkey@0"
I20260811 00:18:20.524619 13474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:20.530499 13474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:20.534425 13474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:20.534726 13474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:20.553333 13474 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:43777
I20260811 00:18:20.553421 13581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:43777 every 8 connection(s)
I20260811 00:18:20.554795 13474 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:18:20.555742 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 13474
I20260811 00:18:20.555938 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:18:20.556474 7544 external_mini_cluster.cc:1428] Setting key bbb1290ab14216d6f3ed791a3acc1920
I20260811 00:18:20.559350 13582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:20.564270 13585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:20.558334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33855 (local address 127.7.94.62:43777)
0811 00:18:20.559133 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:20.559145 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:20.559186 (+ 41us) server_negotiation.cc:407] Connection header received
0811 00:18:20.560879 (+ 1693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:20.560903 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:20.561275 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:20.561615 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:20.562698 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:20.562703 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:20.562722 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:18:20.562735 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:20.562802 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:18:20.562821 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:20.563002 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:20.563145 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 00:18:20.563353 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":362,"threads_started":1}
I20260811 00:18:20.567919 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:18:20.572319 13582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a: Bootstrap starting.
I20260811 00:18:20.575942 13585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:20.573306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37691 (local address 127.7.94.62:43777)
0811 00:18:20.573489 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:20.573495 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:20.573912 (+ 417us) server_negotiation.cc:407] Connection header received
0811 00:18:20.574094 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:20.574101 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:20.574199 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:20.574319 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:20.575240 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:20.575245 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:20.575250 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:18:20.575267 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:20.575311 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:18:20.575330 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:20.575452 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:20.575569 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 00:18:20.575624 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 00:18:20.576200 13582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:20.577585 13582 log.cc:824] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:20.580217 13582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a: No bootstrap required, opened a new log
I20260811 00:18:20.588088 13582 raft_consensus.cc:348] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369645504fdf4454b9666a61324d517a" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43777 } }
I20260811 00:18:20.588433 13582 raft_consensus.cc:374] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:18:20.588506 13582 raft_consensus.cc:729] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 369645504fdf4454b9666a61324d517a, State: Initialized, Role: FOLLOWER
I20260811 00:18:20.588991 13582 consensus_queue.cc:260] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369645504fdf4454b9666a61324d517a" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43777 } }
I20260811 00:18:20.589190 13582 raft_consensus.cc:388] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:18:20.589375 13582 raft_consensus.cc:482] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:18:20.589504 13582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:20.591816 13582 raft_consensus.cc:504] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369645504fdf4454b9666a61324d517a" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43777 } }
I20260811 00:18:20.592382 13582 leader_election.cc:302] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 369645504fdf4454b9666a61324d517a; no voters:
I20260811 00:18:20.592901 13582 leader_election.cc:288] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:18:20.593057 13594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:18:20.593789 13594 raft_consensus.cc:686] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a [term 1 LEADER]: Becoming Leader. State: Replica: 369645504fdf4454b9666a61324d517a, State: Running, Role: LEADER
I20260811 00:18:20.594355 13594 consensus_queue.cc:237] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369645504fdf4454b9666a61324d517a" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43777 } }
I20260811 00:18:20.595079 13582 sys_catalog.cc:564] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:18:20.597129 13595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "369645504fdf4454b9666a61324d517a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369645504fdf4454b9666a61324d517a" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43777 } } }
I20260811 00:18:20.597577 13595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:20.598079 13596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 369645504fdf4454b9666a61324d517a. Latest consensus state: current_term: 1 leader_uuid: "369645504fdf4454b9666a61324d517a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "369645504fdf4454b9666a61324d517a" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43777 } } }
I20260811 00:18:20.598562 13596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:20.600560 13605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:18:20.605922 13605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:18:20.614130 13605 catalog_manager.cc:1347] Generated new cluster ID: 39b2de4b41804342a8c32ce88bcb81dd
I20260811 00:18:20.614252 13605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:18:20.632370 13605 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:18:20.632879 13605 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:18:20.639706 13605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 369645504fdf4454b9666a61324d517a: Generated new TSK 0
I20260811 00:18:20.640733 13605 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:18:20.654632 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 13474
W20260811 00:18:20.655735 13575 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33855
W20260811 00:18:20.656430 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 13474: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:18:20.656554 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 13474
2026-08-11T00:18:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2285 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:18:20 dist-test-slave-8hhm krb5kdc[13625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:18:20 dist-test-slave-8hhm krb5kdc[13625](info): set up 2 sockets
Aug 11 00:18:20 dist-test-slave-8hhm krb5kdc[13625](info): commencing operation
krb5kdc: starting...
W20260811 00:18:22.033735 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.328s user 0.007s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:18:22 dist-test-slave-8hhm krb5kdc[13625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:18:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:18:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:18:22.352612 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:33661
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:33661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:22.694949 13644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:22.695219 13644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:22.695358 13644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:22.702771 13644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:22.702909 13644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:22.703019 13644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:22.703135 13644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:22.703219 13644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:22.712607 13644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:33661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:33661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:22.713963 13644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:22.715262 13644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:22.726924 13656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:22.726929 13657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:22.727135 13659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:22.727964 13644 server_base.cc:1034] running on GCE node
I20260811 00:18:22.728729 13644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:22.730182 13644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:22.731379 13644 hybrid_clock.cc:648] HybridClock initialized: now 1786407502731311 us; error 70 us; skew 500 ppm
I20260811 00:18:22.731756 13644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:22 dist-test-slave-8hhm krb5kdc[13625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407502, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:22.739001 13644 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:18:22.740411 13644 webserver.cc:460] Webserver started at http://127.7.94.62:40985/ using document root <none> and password file <none>
I20260811 00:18:22.741045 13644 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:22.741144 13644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:22.741456 13644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:22.744488 13644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "edf3645d6bca4673aec6ff91b65cfd7f"
format_stamp: "Formatted at 2026-08-11 00:18:22 on dist-test-slave-8hhm"
server_key: "379082e494106ff577351db2d6eb3879"
server_key_iv: "624eb3300e598731f98a38888102a3c3"
server_key_version: "encryptionkey@0"
I20260811 00:18:22.745210 13644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "edf3645d6bca4673aec6ff91b65cfd7f"
format_stamp: "Formatted at 2026-08-11 00:18:22 on dist-test-slave-8hhm"
server_key: "379082e494106ff577351db2d6eb3879"
server_key_iv: "624eb3300e598731f98a38888102a3c3"
server_key_version: "encryptionkey@0"
I20260811 00:18:22.750339 13644 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 00:18:22.753592 13672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:22.754899 13644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:18:22.755028 13644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "edf3645d6bca4673aec6ff91b65cfd7f"
format_stamp: "Formatted at 2026-08-11 00:18:22 on dist-test-slave-8hhm"
server_key: "379082e494106ff577351db2d6eb3879"
server_key_iv: "624eb3300e598731f98a38888102a3c3"
server_key_version: "encryptionkey@0"
I20260811 00:18:22.755177 13644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:22.776021 13644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:22.780377 13644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:22.780611 13644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:22.799563 13644 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:33661
I20260811 00:18:22.799585 13762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:33661 every 8 connection(s)
I20260811 00:18:22.800904 13644 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:18:22.804595 13764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:22.805326 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 13644
I20260811 00:18:22.805529 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:18:22.806034 7544 external_mini_cluster.cc:1428] Setting key 1dbaa8cebe3a45df5d1f3798fcc11253
I20260811 00:18:22.816718 13764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f: Bootstrap starting.
I20260811 00:18:22.819545 13764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f: Neither blocks nor log segments found. Creating new log.
Aug 11 00:18:22 dist-test-slave-8hhm krb5kdc[13625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:18:22.820571 13764 log.cc:824] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:22.824046 13764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f: No bootstrap required, opened a new log
I20260811 00:18:22.829576 13770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:22.807869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59207 (local address 127.7.94.62:33661)
0811 00:18:22.808696 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:22.808708 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:22.808747 (+ 39us) server_negotiation.cc:407] Connection header received
0811 00:18:22.810858 (+ 2111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:22.810881 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:22.811273 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:22.811665 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:22.813200 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:22.814556 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:22.815874 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:22.816561 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:22.816794 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:22.820889 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:22.820899 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:22.820916 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:22.820961 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:22.825038 (+ 4077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:22.825776 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:22.825785 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:22.825794 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:22.826001 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:22.826381 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:22.826387 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:22.826391 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:22.827092 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:22.827426 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:22.827778 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 00:18:22.828219 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":356,"threads_started":1}
I20260811 00:18:22.833465 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:18:22.834280 13764 raft_consensus.cc:348] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf3645d6bca4673aec6ff91b65cfd7f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 33661 } }
I20260811 00:18:22.834615 13764 raft_consensus.cc:374] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:18:22.834738 13764 raft_consensus.cc:729] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edf3645d6bca4673aec6ff91b65cfd7f, State: Initialized, Role: FOLLOWER
I20260811 00:18:22.835278 13764 consensus_queue.cc:260] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf3645d6bca4673aec6ff91b65cfd7f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 33661 } }
I20260811 00:18:22.835482 13764 raft_consensus.cc:388] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:18:22.835656 13764 raft_consensus.cc:482] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:18:22.835819 13764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:22.838241 13764 raft_consensus.cc:504] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf3645d6bca4673aec6ff91b65cfd7f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 33661 } }
I20260811 00:18:22.838774 13764 leader_election.cc:302] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edf3645d6bca4673aec6ff91b65cfd7f; no voters:
I20260811 00:18:22.839419 13764 leader_election.cc:288] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:18:22.839555 13777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:18:22.841465 13764 sys_catalog.cc:564] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:18:22.841974 13777 raft_consensus.cc:686] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f [term 1 LEADER]: Becoming Leader. State: Replica: edf3645d6bca4673aec6ff91b65cfd7f, State: Running, Role: LEADER
I20260811 00:18:22.842528 13777 consensus_queue.cc:237] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf3645d6bca4673aec6ff91b65cfd7f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 33661 } }
I20260811 00:18:22.845676 13781 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader edf3645d6bca4673aec6ff91b65cfd7f. Latest consensus state: current_term: 1 leader_uuid: "edf3645d6bca4673aec6ff91b65cfd7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf3645d6bca4673aec6ff91b65cfd7f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 33661 } } }
I20260811 00:18:22.846258 13781 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:22.846637 13778 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edf3645d6bca4673aec6ff91b65cfd7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf3645d6bca4673aec6ff91b65cfd7f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 33661 } } }
I20260811 00:18:22.847915 13778 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:22.857923 13770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:22.838388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42297 (local address 127.7.94.62:33661)
0811 00:18:22.838551 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:22.838556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:22.839042 (+ 486us) server_negotiation.cc:407] Connection header received
0811 00:18:22.839192 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:22.839205 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:22.839299 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:22.839422 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:22.841708 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:22.842850 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:22.845202 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:22.846204 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:22.846360 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:22.850292 (+ 3932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:22.850302 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:22.850309 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:22.850358 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:22.854929 (+ 4571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:22.855640 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:22.855653 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:22.855657 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:22.855776 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:22.856171 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:22.856177 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:22.856180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:22.856477 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:22.856782 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:22.857109 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 00:18:22.857363 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 00:18:22.858333 13790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:18:22.864048 13790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:18:22.872932 13790 catalog_manager.cc:1347] Generated new cluster ID: ed03ecc952714070bc2ee4335b3c2f40
I20260811 00:18:22.873035 13790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:18:22.894373 13790 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:18:22.896082 13790 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:18:22.904697 13790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edf3645d6bca4673aec6ff91b65cfd7f: Generated new TSK 0
I20260811 00:18:22.905614 13790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:18:22.915498 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 13644
I20260811 00:18:22.916841 13755 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:18:23.007707 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 13644
2026-08-11T00:18:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2345 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:18:23 dist-test-slave-8hhm krb5kdc[13807](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:18:23 dist-test-slave-8hhm krb5kdc[13807](info): set up 2 sockets
Aug 11 00:18:23 dist-test-slave-8hhm krb5kdc[13807](info): commencing operation
krb5kdc: starting...
W20260811 00:18:24.375990 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.325s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:18:24 dist-test-slave-8hhm krb5kdc[13807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:18:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:18:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:18:24.696473 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:34263
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:38333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:34263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:25.038225 13829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:25.038528 13829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:25.038702 13829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:25.045706 13829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:25.045845 13829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:25.045930 13829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:25.046026 13829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:25.046121 13829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:25.055807 13829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:38333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:34263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:34263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:25.057139 13829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:25.058499 13829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:25.070932 13836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:25.071007 13839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:25.070899 13841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:25.074584 13829 server_base.cc:1034] running on GCE node
I20260811 00:18:25.075345 13829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:25.076695 13829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:25.077929 13829 hybrid_clock.cc:648] HybridClock initialized: now 1786407505077899 us; error 51 us; skew 500 ppm
I20260811 00:18:25.078298 13829 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:25 dist-test-slave-8hhm krb5kdc[13807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407505, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:25.082471 13829 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:18:25.083917 13829 webserver.cc:460] Webserver started at http://127.7.94.62:33927/ using document root <none> and password file <none>
I20260811 00:18:25.084633 13829 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:25.084784 13829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:25.085109 13829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:25.088337 13829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "0b3a663fd6ab4d0e9e9c41c372c1b15d"
format_stamp: "Formatted at 2026-08-11 00:18:25 on dist-test-slave-8hhm"
server_key: "f1cedfd948126755b4dcc8ccc8ea898b"
server_key_iv: "c94d9b0d61ab3c94a1fe4072034c882a"
server_key_version: "encryptionkey@0"
I20260811 00:18:25.089121 13829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "0b3a663fd6ab4d0e9e9c41c372c1b15d"
format_stamp: "Formatted at 2026-08-11 00:18:25 on dist-test-slave-8hhm"
server_key: "f1cedfd948126755b4dcc8ccc8ea898b"
server_key_iv: "c94d9b0d61ab3c94a1fe4072034c882a"
server_key_version: "encryptionkey@0"
I20260811 00:18:25.094292 13829 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 00:18:25.097710 13859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:25.098881 13829 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 00:18:25.099066 13829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "0b3a663fd6ab4d0e9e9c41c372c1b15d"
format_stamp: "Formatted at 2026-08-11 00:18:25 on dist-test-slave-8hhm"
server_key: "f1cedfd948126755b4dcc8ccc8ea898b"
server_key_iv: "c94d9b0d61ab3c94a1fe4072034c882a"
server_key_version: "encryptionkey@0"
I20260811 00:18:25.099254 13829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:25.127000 13829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:25.130959 13829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:25.131269 13829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:25.150187 13829 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:34263
I20260811 00:18:25.150205 13940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:34263 every 8 connection(s)
I20260811 00:18:25.151521 13829 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:18:25.155256 13941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:25.157428 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 13829
I20260811 00:18:25.157606 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:18:25.158164 7544 external_mini_cluster.cc:1428] Setting key dbe4f5f362384d7f9ef6e2e6e2c0a3a1
I20260811 00:18:25.169431 13941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d: Bootstrap starting.
I20260811 00:18:25.171248 13944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:25.159985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46693 (local address 127.7.94.62:34263)
0811 00:18:25.160809 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:25.160820 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:25.160861 (+ 41us) server_negotiation.cc:407] Connection header received
0811 00:18:25.162508 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:25.162533 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:25.162907 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:25.163258 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:25.163828 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:25.165046 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:25.166877 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:25.167564 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:25.167687 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:25.168948 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:25.168956 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:25.168968 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:18:25.169600 (+ 632us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:25.169678 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:18:25.169705 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:25.169794 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:25.170019 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 00:18:25.170298 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":389,"threads_started":1}
I20260811 00:18:25.172703 13941 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:25.173907 13941 log.cc:824] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:25.174886 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:18:25.176636 13941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d: No bootstrap required, opened a new log
I20260811 00:18:25.185602 13944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:25.179177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52549 (local address 127.7.94.62:34263)
0811 00:18:25.179332 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:25.179337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:25.179839 (+ 502us) server_negotiation.cc:407] Connection header received
0811 00:18:25.179991 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:25.179996 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:25.180069 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:25.180184 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:25.180622 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:25.181759 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:25.182978 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:25.183696 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:25.183825 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:25.184804 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:25.184809 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:25.184813 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:18:25.184830 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:25.184884 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:18:25.184901 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:25.184985 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:25.185122 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 00:18:25.185250 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 00:18:25.185384 13941 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3a663fd6ab4d0e9e9c41c372c1b15d" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34263 } }
I20260811 00:18:25.185966 13941 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:18:25.186101 13941 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b3a663fd6ab4d0e9e9c41c372c1b15d, State: Initialized, Role: FOLLOWER
I20260811 00:18:25.186740 13941 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3a663fd6ab4d0e9e9c41c372c1b15d" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34263 } }
I20260811 00:18:25.187026 13941 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:18:25.187177 13941 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:18:25.187350 13941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:25.189685 13941 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3a663fd6ab4d0e9e9c41c372c1b15d" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34263 } }
I20260811 00:18:25.190145 13941 leader_election.cc:302] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b3a663fd6ab4d0e9e9c41c372c1b15d; no voters:
I20260811 00:18:25.190697 13941 leader_election.cc:288] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:18:25.190828 13958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:18:25.192265 13958 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d [term 1 LEADER]: Becoming Leader. State: Replica: 0b3a663fd6ab4d0e9e9c41c372c1b15d, State: Running, Role: LEADER
I20260811 00:18:25.192764 13958 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3a663fd6ab4d0e9e9c41c372c1b15d" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34263 } }
I20260811 00:18:25.192937 13941 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:18:25.195861 13960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b3a663fd6ab4d0e9e9c41c372c1b15d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3a663fd6ab4d0e9e9c41c372c1b15d" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34263 } } }
I20260811 00:18:25.196313 13960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:25.195837 13961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b3a663fd6ab4d0e9e9c41c372c1b15d. Latest consensus state: current_term: 1 leader_uuid: "0b3a663fd6ab4d0e9e9c41c372c1b15d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3a663fd6ab4d0e9e9c41c372c1b15d" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 34263 } } }
I20260811 00:18:25.196712 13961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:25.197607 13972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:18:25.202909 13972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:18:25.211383 13972 catalog_manager.cc:1347] Generated new cluster ID: 438f2552f4ab4a10afb9024a597a3266
I20260811 00:18:25.211485 13972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:18:25.248576 13972 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:18:25.250347 13972 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:18:25.262427 13972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b3a663fd6ab4d0e9e9c41c372c1b15d: Generated new TSK 0
I20260811 00:18:25.263334 13972 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:18:25.277949 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 13829
W20260811 00:18:25.279201 13931 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46693
W20260811 00:18:25.279943 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 13829: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:18:25.280097 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 13829
2026-08-11T00:18:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2274 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:18:25 dist-test-slave-8hhm krb5kdc[13989](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:18:25 dist-test-slave-8hhm krb5kdc[13989](info): set up 2 sockets
Aug 11 00:18:25 dist-test-slave-8hhm krb5kdc[13989](info): commencing operation
krb5kdc: starting...
W20260811 00:18:26.651190 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.326s user 0.005s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:18:26 dist-test-slave-8hhm krb5kdc[13989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:18:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:18:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:18:26.978780 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:40453
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:40889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:40453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:27.323063 14010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:27.323444 14010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:27.323616 14010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:27.330997 14010 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:27.331167 14010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:27.331278 14010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:27.331391 14010 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:27.331471 14010 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:27.341302 14010 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:40889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:40453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:40453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:27.342626 14010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:27.344033 14010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:27.356046 14017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:27.356076 14018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:27.356283 14023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:27.356889 14010 server_base.cc:1034] running on GCE node
I20260811 00:18:27.357731 14010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:27.359038 14010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:27.360270 14010 hybrid_clock.cc:648] HybridClock initialized: now 1786407507360231 us; error 52 us; skew 500 ppm
I20260811 00:18:27.360599 14010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:27 dist-test-slave-8hhm krb5kdc[13989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407507, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:27.363698 14010 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:18:27.364984 14010 webserver.cc:460] Webserver started at http://127.7.94.62:35681/ using document root <none> and password file <none>
I20260811 00:18:27.365675 14010 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:27.365782 14010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:27.366025 14010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:27.368841 14010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "c7db5dab4ca347a48e92009af79f7ad8"
format_stamp: "Formatted at 2026-08-11 00:18:27 on dist-test-slave-8hhm"
server_key: "0961965ca9d00a2ff195c41e58844b4e"
server_key_iv: "7f72cfe320f3ac7dc00365ad3a5241a2"
server_key_version: "encryptionkey@0"
I20260811 00:18:27.369616 14010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "c7db5dab4ca347a48e92009af79f7ad8"
format_stamp: "Formatted at 2026-08-11 00:18:27 on dist-test-slave-8hhm"
server_key: "0961965ca9d00a2ff195c41e58844b4e"
server_key_iv: "7f72cfe320f3ac7dc00365ad3a5241a2"
server_key_version: "encryptionkey@0"
I20260811 00:18:27.374511 14010 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 00:18:27.377851 14035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:27.379073 14010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:18:27.379201 14010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "c7db5dab4ca347a48e92009af79f7ad8"
format_stamp: "Formatted at 2026-08-11 00:18:27 on dist-test-slave-8hhm"
server_key: "0961965ca9d00a2ff195c41e58844b4e"
server_key_iv: "7f72cfe320f3ac7dc00365ad3a5241a2"
server_key_version: "encryptionkey@0"
I20260811 00:18:27.379351 14010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:27.384704 14010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:27.388446 14010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:27.388695 14010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:27.407753 14010 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:40453
I20260811 00:18:27.407797 14115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:40453 every 8 connection(s)
I20260811 00:18:27.409256 14010 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:18:27.411199 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 14010
I20260811 00:18:27.411389 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:18:27.411996 7544 external_mini_cluster.cc:1428] Setting key 234bbc7683fa2005dbbfee3472ae6164
I20260811 00:18:27.414043 14117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:27.420863 14121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:27.413848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52223 (local address 127.7.94.62:40453)
0811 00:18:27.414679 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:27.414695 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:27.414742 (+ 47us) server_negotiation.cc:407] Connection header received
0811 00:18:27.416891 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:27.416919 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:27.417448 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:27.417906 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:27.418904 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:27.418911 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:27.418933 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:18:27.418961 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:27.419039 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:18:27.419069 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:27.419283 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:27.419426 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 00:18:27.419639 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":346,"threads_started":1}
I20260811 00:18:27.424494 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:18:27.428220 14117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8: Bootstrap starting.
I20260811 00:18:27.432444 14117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:27.432672 14121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:27.429936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56905 (local address 127.7.94.62:40453)
0811 00:18:27.430131 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:27.430139 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:27.430684 (+ 545us) server_negotiation.cc:407] Connection header received
0811 00:18:27.430847 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:27.430850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:27.430902 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:27.430980 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:27.431936 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:27.431940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:27.431943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 00:18:27.431957 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:27.431996 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 00:18:27.432010 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:27.432096 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:27.432253 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 00:18:27.432299 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 00:18:27.433763 14117 log.cc:824] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:27.436343 14117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8: No bootstrap required, opened a new log
I20260811 00:18:27.443817 14117 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7db5dab4ca347a48e92009af79f7ad8" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40453 } }
I20260811 00:18:27.444198 14117 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 00:18:27.444301 14117 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7db5dab4ca347a48e92009af79f7ad8, State: Initialized, Role: FOLLOWER
I20260811 00:18:27.444844 14117 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7db5dab4ca347a48e92009af79f7ad8" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40453 } }
I20260811 00:18:27.445086 14117 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 00:18:27.445262 14117 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 00:18:27.445451 14117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:27.447614 14117 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7db5dab4ca347a48e92009af79f7ad8" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40453 } }
I20260811 00:18:27.448108 14117 leader_election.cc:302] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7db5dab4ca347a48e92009af79f7ad8; no voters:
I20260811 00:18:27.448730 14117 leader_election.cc:288] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 00:18:27.448866 14131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:18:27.450009 14131 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8 [term 1 LEADER]: Becoming Leader. State: Replica: c7db5dab4ca347a48e92009af79f7ad8, State: Running, Role: LEADER
I20260811 00:18:27.450517 14131 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7db5dab4ca347a48e92009af79f7ad8" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40453 } }
I20260811 00:18:27.450924 14117 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:18:27.453547 14134 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7db5dab4ca347a48e92009af79f7ad8. Latest consensus state: current_term: 1 leader_uuid: "c7db5dab4ca347a48e92009af79f7ad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7db5dab4ca347a48e92009af79f7ad8" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40453 } } }
I20260811 00:18:27.454046 14134 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:27.453640 14133 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7db5dab4ca347a48e92009af79f7ad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7db5dab4ca347a48e92009af79f7ad8" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 40453 } } }
I20260811 00:18:27.454455 14133 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:27.457441 14147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 00:18:27.462342 14147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:18:27.471205 14147 catalog_manager.cc:1347] Generated new cluster ID: 5bf1d3685a7a4b0bb0d43ed5287e5205
I20260811 00:18:27.471328 14147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:18:27.489614 14147 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:18:27.490073 14147 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:18:27.501314 14147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7db5dab4ca347a48e92009af79f7ad8: Generated new TSK 0
I20260811 00:18:27.502244 14147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:18:27.511792 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 14010
W20260811 00:18:27.512897 14104 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52223
W20260811 00:18:27.513610 7544 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 14010: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 00:18:27.513768 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 14010
2026-08-11T00:18:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2234 ms)
[----------] 18 tests from AuthTokenIssuingTest (41159 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:18:27 dist-test-slave-8hhm krb5kdc[14169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:18:27 dist-test-slave-8hhm krb5kdc[14169](info): set up 2 sockets
Aug 11 00:18:27 dist-test-slave-8hhm krb5kdc[14169](info): commencing operation
krb5kdc: starting...
W20260811 00:18:28.865543 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.003s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:18:29 dist-test-slave-8hhm krb5kdc[14169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:18:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:18:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:18:29.200137 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:41259
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:41259,127.7.94.61:46721,127.7.94.60:41571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:29.542424 14191 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 00:18:29.542809 14191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:29.542939 14191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:29.543078 14191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:29.550236 14191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:29.550372 14191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:29.550477 14191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:29.550582 14191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:29.550673 14191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:29.560304 14191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:41259,127.7.94.61:46721,127.7.94.60:41571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:41259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:29.561662 14191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:29.563001 14191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:29.574519 14203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:29.574487 14204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:29.574756 14206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:29.575742 14191 server_base.cc:1034] running on GCE node
I20260811 00:18:29.576496 14191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:29.577852 14191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:29.579107 14191 hybrid_clock.cc:648] HybridClock initialized: now 1786407509579050 us; error 31 us; skew 500 ppm
I20260811 00:18:29.579423 14191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:29 dist-test-slave-8hhm krb5kdc[14169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407509, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:29.587595 14191 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:18:29.589105 14191 webserver.cc:460] Webserver started at http://127.7.94.62:44885/ using document root <none> and password file <none>
I20260811 00:18:29.589869 14191 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:29.590003 14191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:29.590340 14191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:29.593369 14191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "85e5d0c211f24aae93187fa6adb60f76"
format_stamp: "Formatted at 2026-08-11 00:18:29 on dist-test-slave-8hhm"
server_key: "99a5381d0c7a395e80bcaffcd0e1b3de"
server_key_iv: "3ac8c687b3b72469c85877ec4357f516"
server_key_version: "encryptionkey@0"
I20260811 00:18:29.594173 14191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "85e5d0c211f24aae93187fa6adb60f76"
format_stamp: "Formatted at 2026-08-11 00:18:29 on dist-test-slave-8hhm"
server_key: "99a5381d0c7a395e80bcaffcd0e1b3de"
server_key_iv: "3ac8c687b3b72469c85877ec4357f516"
server_key_version: "encryptionkey@0"
I20260811 00:18:29.599397 14191 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:18:29.602720 14218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:29.603928 14191 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 00:18:29.604125 14191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "85e5d0c211f24aae93187fa6adb60f76"
format_stamp: "Formatted at 2026-08-11 00:18:29 on dist-test-slave-8hhm"
server_key: "99a5381d0c7a395e80bcaffcd0e1b3de"
server_key_iv: "3ac8c687b3b72469c85877ec4357f516"
server_key_version: "encryptionkey@0"
I20260811 00:18:29.604336 14191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:29.619292 14191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:29.623061 14191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:29.623337 14191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:29.641829 14191 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:41259
I20260811 00:18:29.641834 14299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:41259 every 8 connection(s)
I20260811 00:18:29.643301 14191 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:18:29.646981 14302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:29.649996 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 14191
I20260811 00:18:29.650236 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:18:29.651077 7544 external_mini_cluster.cc:1428] Setting key b38f123726501374aa9685d6facb99f4
I20260811 00:18:29.655802 14302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:18:29 dist-test-slave-8hhm krb5kdc[14169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407509, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:18:29.681528 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:29.660533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40056 (local address 127.7.94.62:41259)
0811 00:18:29.664266 (+ 3733us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:29.664271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:29.664299 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:18:29.664418 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:29.664423 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:29.664628 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:29.664821 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:29.666220 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:29.667427 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:29.669687 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:29.670213 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:29.670326 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:29.674393 (+ 4067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:29.674400 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:29.674410 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:29.674436 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:29.677098 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:29.677805 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:29.677812 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:29.677814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:29.677953 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:29.678351 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:29.678370 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:29.678373 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:29.678863 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:29.679079 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:29.679601 (+ 522us) server_negotiation.cc:299] Negotiation successful
0811 00:18:29.679984 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":3565,"thread_start_us":3451,"threads_started":1}
I20260811 00:18:29.681548 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:29.659315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41259 (local address 127.0.0.1:40056)
0811 00:18:29.660107 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:29.660158 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:29.663373 (+ 3215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:29.664844 (+ 1471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:29.664850 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:29.665537 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:29.665988 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:29.666015 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:29.667589 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:29.667597 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:29.669553 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:29.669570 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:29.670437 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:29.670446 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:29.670670 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:29.671413 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:29.671446 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:29.674188 (+ 2742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:29.677253 (+ 3065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:29.677264 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:29.677275 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:29.677670 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:29.678062 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:29.678068 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:29.678073 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:29.678240 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:29.678993 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:29.679001 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:29.679229 (+ 228us) client_negotiation.cc:769] Sending connection context
0811 00:18:29.679463 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 00:18:29.680100 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":193,"thread_start_us":347,"threads_started":1}
WARNING: no policy specified for kudu/127.7.94.61@KRBTEST.COM; defaulting to no policy
I20260811 00:18:29.685390 14302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:18:29.686333 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:29.685966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:46721 (local address 127.0.0.1:51946)
0811 00:18:29.686106 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:29.686214 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:46721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 00:18:29.686565 14219 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.94.61:46721: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.61@KRBTEST.COM" created.
I20260811 00:18:29.688205 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:29.687869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:46721 (local address 127.0.0.1:51948)
0811 00:18:29.688009 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:29.688083 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:46721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 00:18:29.688378 14302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.61:46721: Network error: Client connection negotiation failed: client connection to 127.7.94.61:46721: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.61.keytab.
Entry for principal kudu/127.7.94.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.61.keytab.
I20260811 00:18:29.724267 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.61:46721
--webserver_interface=127.7.94.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.61.keytab
--principal=kudu/127.7.94.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:41259,127.7.94.61:46721,127.7.94.60:41571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:18:29.737668 14302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } attempt: 1
I20260811 00:18:29.738852 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:29.738392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:46721 (local address 127.0.0.1:51960)
0811 00:18:29.738591 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:29.738685 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:46721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 00:18:29.739889 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:29.739527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:46721 (local address 127.0.0.1:51976)
0811 00:18:29.739694 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:29.739763 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:46721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 00:18:29.740067 14302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.61:46721: Network error: Client connection negotiation failed: client connection to 127.7.94.61:46721: connect: Connection refused (error 111)
I20260811 00:18:29.808331 14302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } attempt: 2
I20260811 00:18:29.809615 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:29.809103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:46721 (local address 127.0.0.1:51990)
0811 00:18:29.809393 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:29.809480 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:46721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 00:18:29.810693 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:29.810332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:46721 (local address 127.0.0.1:51992)
0811 00:18:29.810483 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:29.810559 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:46721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 00:18:29.810868 14302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.61:46721: Network error: Client connection negotiation failed: client connection to 127.7.94.61:46721: connect: Connection refused (error 111)
I20260811 00:18:29.902109 14302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } attempt: 3
I20260811 00:18:29.903240 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:29.902814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:46721 (local address 127.0.0.1:51994)
0811 00:18:29.903009 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:29.903102 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:46721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 00:18:29.904261 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:29.903928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:46721 (local address 127.0.0.1:52000)
0811 00:18:29.904071 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:29.904132 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:46721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 00:18:29.904439 14302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.61:46721: Network error: Client connection negotiation failed: client connection to 127.7.94.61:46721: connect: Connection refused (error 111)
I20260811 00:18:30.047681 14302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } attempt: 4
I20260811 00:18:30.048826 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.048317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:46721 (local address 127.0.0.1:52016)
0811 00:18:30.048611 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.048703 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:46721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260811 00:18:30.049880 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.049539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:46721 (local address 127.0.0.1:52020)
0811 00:18:30.049676 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.049759 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:46721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 00:18:30.050055 14302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.61:46721: Network error: Client connection negotiation failed: client connection to 127.7.94.61:46721: connect: Connection refused (error 111)
W20260811 00:18:30.066291 14316 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 00:18:30.066650 14316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:30.066776 14316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:30.066915 14316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:30.074048 14316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:30.074167 14316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:30.074291 14316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:30.074380 14316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:30.074491 14316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:30.084656 14316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:41259,127.7.94.61:46721,127.7.94.60:41571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.61.keytab
--principal=kudu/127.7.94.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.61:46721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.94.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:30.085999 14316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:30.087301 14316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:30.099368 14329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:30.099550 14326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:30.100209 14327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:30.101039 14316 server_base.cc:1034] running on GCE node
I20260811 00:18:30.101878 14316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:30.103282 14316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:30.104516 14316 hybrid_clock.cc:648] HybridClock initialized: now 1786407510104459 us; error 37 us; skew 500 ppm
I20260811 00:18:30.104883 14316 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:30 dist-test-slave-8hhm krb5kdc[14169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407510, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:30.108009 14316 init.cc:377] Logged in from keytab as kudu/127.7.94.61@KRBTEST.COM (short username kudu)
I20260811 00:18:30.109467 14316 webserver.cc:460] Webserver started at http://127.7.94.61:46239/ using document root <none> and password file <none>
I20260811 00:18:30.110110 14316 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:30.110247 14316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:30.110554 14316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:30.113440 14316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-1/data/instance:
uuid: "145b7c17e33f4ecda830340911cf39f5"
format_stamp: "Formatted at 2026-08-11 00:18:30 on dist-test-slave-8hhm"
server_key: "13e2afad6869172a28a8d8e68cbf5c7b"
server_key_iv: "677d57cf65ef1724278ac5889df7f2ad"
server_key_version: "encryptionkey@0"
I20260811 00:18:30.114209 14316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-1/wal/instance:
uuid: "145b7c17e33f4ecda830340911cf39f5"
format_stamp: "Formatted at 2026-08-11 00:18:30 on dist-test-slave-8hhm"
server_key: "13e2afad6869172a28a8d8e68cbf5c7b"
server_key_iv: "677d57cf65ef1724278ac5889df7f2ad"
server_key_version: "encryptionkey@0"
I20260811 00:18:30.119127 14316 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 00:18:30.122319 14337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:30.123546 14316 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 00:18:30.123718 14316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-1/wal
uuid: "145b7c17e33f4ecda830340911cf39f5"
format_stamp: "Formatted at 2026-08-11 00:18:30 on dist-test-slave-8hhm"
server_key: "13e2afad6869172a28a8d8e68cbf5c7b"
server_key_iv: "677d57cf65ef1724278ac5889df7f2ad"
server_key_version: "encryptionkey@0"
I20260811 00:18:30.123917 14316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:30.146564 14316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:30.150753 14316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:30.151014 14316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:30.169697 14316 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.61:46721
I20260811 00:18:30.169724 14427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.61:46721 every 8 connection(s)
I20260811 00:18:30.171001 14316 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-1/data/info.pb
I20260811 00:18:30.174515 14428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:30.176046 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 14316
I20260811 00:18:30.176235 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-1/wal/instance
I20260811 00:18:30.176820 7544 external_mini_cluster.cc:1428] Setting key 39c8858742433d000282f2cca6957651
I20260811 00:18:30.184166 14428 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:18:30 dist-test-slave-8hhm krb5kdc[14169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407510, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.61@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:18:30.207684 14430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.187628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40062 (local address 127.7.94.62:41259)
0811 00:18:30.188220 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:30.188226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:30.189097 (+ 871us) server_negotiation.cc:407] Connection header received
0811 00:18:30.192007 (+ 2910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:30.192012 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:30.192079 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:30.192176 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:30.193947 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.195214 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.196649 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.197410 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.197549 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.201398 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:30.201403 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:30.201406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:30.201443 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:30.204180 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.205136 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.205140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.205142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.205318 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.205866 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.205869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.205871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.206140 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:30.206249 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:30.207188 (+ 939us) server_negotiation.cc:299] Negotiation successful
0811 00:18:30.207369 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":329,"threads_started":1}
I20260811 00:18:30.209236 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.188071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41259 (local address 127.0.0.1:40062)
0811 00:18:30.188918 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.188969 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:30.191693 (+ 2724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:30.192452 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:30.192467 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:30.193092 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:30.193723 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:30.193741 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.195356 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.195360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:30.196505 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:30.196517 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.197602 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.197608 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:30.197865 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.198563 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:30.198585 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:30.201133 (+ 2548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:30.204344 (+ 3211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:30.204355 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:30.204373 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:30.205020 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:30.205421 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:30.205429 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:30.205435 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:30.205731 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:30.206266 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:30.206276 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:30.206750 (+ 474us) client_negotiation.cc:769] Sending connection context
0811 00:18:30.207089 (+ 339us) client_negotiation.cc:240] Negotiation successful
0811 00:18:30.207679 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":323,"threads_started":1}
I20260811 00:18:30.211090 14428 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.94.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.60@KRBTEST.COM" created.
Aug 11 00:18:30 dist-test-slave-8hhm krb5kdc[14169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407510, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.61@KRBTEST.COM for kudu/127.7.94.61@KRBTEST.COM
I20260811 00:18:30.225411 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.211804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:46721 (local address 127.0.0.1:52026)
0811 00:18:30.212035 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.212055 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:30.212162 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:30.213316 (+ 1154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:30.213323 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:30.213690 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:30.213831 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:30.213839 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.215340 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.215343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:30.216427 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:30.216439 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.217247 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.217254 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:30.217429 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.218041 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:30.218057 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:30.219985 (+ 1928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:30.222975 (+ 2990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:30.222981 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:30.222988 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:30.223340 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:30.223706 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:30.223711 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:30.223714 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:30.223861 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:30.224422 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:30.224428 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:30.224578 (+ 150us) client_negotiation.cc:769] Sending connection context
0811 00:18:30.224714 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 00:18:30.224991 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 00:18:30.225514 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.212120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52026 (local address 127.7.94.61:46721)
0811 00:18:30.212850 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:30.212856 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:30.212881 (+ 25us) server_negotiation.cc:407] Connection header received
0811 00:18:30.212921 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:30.212927 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:30.213126 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:30.213302 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:30.214005 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.215148 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.216558 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.217068 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.217203 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.220184 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:30.220194 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:30.220206 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:30.220232 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:30.222839 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.223474 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.223480 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.223482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.223584 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.223960 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.223966 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.223970 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.224284 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:30.224508 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:30.224849 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 00:18:30.225106 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":592,"thread_start_us":423,"threads_started":1}
I20260811 00:18:30.228061 14428 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:18:30.229208 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.228710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35682)
0811 00:18:30.228882 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.229039 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 00:18:30.229641 14341 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
I20260811 00:18:30.231233 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.230925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35696)
0811 00:18:30.231056 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.231136 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 00:18:30.231449 14428 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:41571: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.60.keytab.
Entry for principal kudu/127.7.94.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.60.keytab.
I20260811 00:18:30.255908 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.60:41571
--webserver_interface=127.7.94.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.60.keytab
--principal=kudu/127.7.94.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:41053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:41259,127.7.94.61:46721,127.7.94.60:41571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:18:30.280814 14428 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } attempt: 1
I20260811 00:18:30.282032 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.281572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35698)
0811 00:18:30.281754 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.281845 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 00:18:30.283038 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.282694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35714)
0811 00:18:30.282841 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.282930 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 00:18:30.283301 14428 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:41571: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
I20260811 00:18:30.349318 14302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } attempt: 5
I20260811 00:18:30.351547 14428 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } attempt: 2
I20260811 00:18:30.352569 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.352087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35726)
0811 00:18:30.352271 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.352379 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 00:18:30.353682 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.353334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35736)
0811 00:18:30.353477 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.353565 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 00:18:30.353932 14428 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:41571: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
Aug 11 00:18:30 dist-test-slave-8hhm krb5kdc[14169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407509, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for kudu/127.7.94.61@KRBTEST.COM
I20260811 00:18:30.364034 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.350141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:46721 (local address 127.0.0.1:52042)
0811 00:18:30.350345 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.350371 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:30.350532 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:30.350929 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:30.350937 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:30.351353 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:30.351542 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:30.351552 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.352943 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.352951 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:30.354407 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:30.354428 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.355157 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.355165 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:30.355371 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.356026 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:30.356051 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:30.358583 (+ 2532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:30.361687 (+ 3104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:30.361693 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:30.361697 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:30.362090 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:30.362447 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:30.362452 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:30.362456 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:30.362596 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:30.363105 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:30.363110 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:30.363259 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 00:18:30.363423 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 00:18:30.363691 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 00:18:30.364033 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.350368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52042 (local address 127.7.94.61:46721)
0811 00:18:30.350595 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:30.350604 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:30.350627 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:18:30.350699 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:30.350703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:30.350770 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:30.350882 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:30.351718 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.352745 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.354568 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.354989 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.355093 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.358780 (+ 3687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:30.358785 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:30.358788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:30.358818 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:30.361559 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.362205 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.362211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.362215 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.362337 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.362704 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.362710 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.362714 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.362984 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:30.363136 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:30.363541 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 00:18:30.363741 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 00:18:30.365427 14302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:18:30.366508 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.366132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35738)
0811 00:18:30.366296 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.366375 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 00:18:30.367475 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.367151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35746)
0811 00:18:30.367293 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.367364 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 00:18:30.367707 14302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:41571: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
I20260811 00:18:30.419062 14302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } attempt: 1
I20260811 00:18:30.420351 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.419840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35762)
0811 00:18:30.420072 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.420169 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 00:18:30.421475 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.421101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35772)
0811 00:18:30.421280 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.421356 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 00:18:30.421700 14302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:41571: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
I20260811 00:18:30.445187 14428 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } attempt: 3
I20260811 00:18:30.446370 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.445878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35786)
0811 00:18:30.446125 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.446217 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 00:18:30.447417 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.447019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35802)
0811 00:18:30.447236 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.447312 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260811 00:18:30.447659 14428 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:41571: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
I20260811 00:18:30.489943 14302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } attempt: 2
I20260811 00:18:30.491225 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.490703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35816)
0811 00:18:30.490948 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.491065 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 00:18:30.492386 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.492046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35820)
0811 00:18:30.492199 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.492266 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 00:18:30.492630 14302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:41571: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
I20260811 00:18:30.590886 14428 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } attempt: 4
I20260811 00:18:30.592141 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.591629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35830)
0811 00:18:30.591879 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.591982 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 00:18:30.593343 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.592873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35844)
0811 00:18:30.593090 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.593176 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260811 00:18:30.593611 14428 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:41571: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
W20260811 00:18:30.596588 14440 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 00:18:30.596908 14440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:30.597005 14440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:30.597136 14440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 00:18:30.598834 14302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } attempt: 3
I20260811 00:18:30.599751 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.599336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35852)
0811 00:18:30.599497 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.599601 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 00:18:30.600773 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.600401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35856)
0811 00:18:30.600596 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.600673 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260811 00:18:30.600977 14302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:41571: Network error: Client connection negotiation failed: client connection to 127.7.94.60:41571: connect: Connection refused (error 111)
W20260811 00:18:30.604744 14440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:30.604818 14440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:30.604918 14440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:30.604986 14440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:30.605088 14440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:30.615319 14440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:41053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:41259,127.7.94.61:46721,127.7.94.60:41571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.60.keytab
--principal=kudu/127.7.94.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.60:41571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.94.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:30.616631 14440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:30.618002 14440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:30.629709 14445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:30.629650 14450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:30.629741 14446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:30.630558 14440 server_base.cc:1034] running on GCE node
I20260811 00:18:30.631340 14440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:30.632656 14440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:30.633894 14440 hybrid_clock.cc:648] HybridClock initialized: now 1786407510633823 us; error 43 us; skew 500 ppm
I20260811 00:18:30.634218 14440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:30 dist-test-slave-8hhm krb5kdc[14169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407510, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:30.637310 14440 init.cc:377] Logged in from keytab as kudu/127.7.94.60@KRBTEST.COM (short username kudu)
I20260811 00:18:30.638736 14440 webserver.cc:460] Webserver started at http://127.7.94.60:45597/ using document root <none> and password file <none>
I20260811 00:18:30.639379 14440 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:30.639505 14440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:30.639827 14440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:30.642704 14440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-2/data/instance:
uuid: "193f1bd425994c5bba235db3a97c8a98"
format_stamp: "Formatted at 2026-08-11 00:18:30 on dist-test-slave-8hhm"
server_key: "2522d9d46b7d6e361670d3f444562430"
server_key_iv: "8c072fc5e59e4777ddb1708f2faa29f3"
server_key_version: "encryptionkey@0"
I20260811 00:18:30.643448 14440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-2/wal/instance:
uuid: "193f1bd425994c5bba235db3a97c8a98"
format_stamp: "Formatted at 2026-08-11 00:18:30 on dist-test-slave-8hhm"
server_key: "2522d9d46b7d6e361670d3f444562430"
server_key_iv: "8c072fc5e59e4777ddb1708f2faa29f3"
server_key_version: "encryptionkey@0"
I20260811 00:18:30.648595 14440 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 00:18:30.651952 14462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:30.653177 14440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:18:30.653407 14440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-2/wal
uuid: "193f1bd425994c5bba235db3a97c8a98"
format_stamp: "Formatted at 2026-08-11 00:18:30 on dist-test-slave-8hhm"
server_key: "2522d9d46b7d6e361670d3f444562430"
server_key_iv: "8c072fc5e59e4777ddb1708f2faa29f3"
server_key_version: "encryptionkey@0"
I20260811 00:18:30.653631 14440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:30.682330 14440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:30.686535 14440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:30.686820 14440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:30.706607 14440 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.60:41571
I20260811 00:18:30.706621 14560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.60:41571 every 8 connection(s)
I20260811 00:18:30.708001 14440 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-2/data/info.pb
I20260811 00:18:30.711839 14561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:30.715281 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 14440
I20260811 00:18:30.715495 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786407133955383-7544-0/minicluster-data/master-2/wal/instance
I20260811 00:18:30.715935 7544 external_mini_cluster.cc:1428] Setting key 0f08f3fe4157441c3c5af9de6e7c0e1a
I20260811 00:18:30.719444 14561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:18:30 dist-test-slave-8hhm krb5kdc[14169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.60@KRBTEST.COM
Aug 11 00:18:30 dist-test-slave-8hhm krb5kdc[14169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407510, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.60@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:18:30.739457 14568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.723328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40066 (local address 127.7.94.62:41259)
0811 00:18:30.723844 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:30.723850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:30.724381 (+ 531us) server_negotiation.cc:407] Connection header received
0811 00:18:30.724594 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:30.724605 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:30.724679 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:30.724769 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:30.726433 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.727587 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.729511 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.729902 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.730006 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.733852 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:30.733857 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:30.733860 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:30.733899 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:30.736457 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.737106 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.737110 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.737112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.737291 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.737723 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.737726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.737728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.737979 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:30.738088 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:30.738802 (+ 714us) server_negotiation.cc:299] Negotiation successful
0811 00:18:30.739020 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":231,"threads_started":1}
I20260811 00:18:30.740592 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.717927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35866 (local address 127.7.94.60:41571)
0811 00:18:30.718765 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:30.718779 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:30.718829 (+ 50us) server_negotiation.cc:407] Connection header received
0811 00:18:30.720618 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:30.720651 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:30.721201 (+ 550us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:30.721671 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:30.723022 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.725119 (+ 2097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.727114 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.728042 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.728265 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.733283 (+ 5018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:30.733293 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:30.733305 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:30.733340 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:30.736384 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.737032 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.737044 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.737050 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.737250 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.737674 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.737679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.737682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.738262 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:30.738598 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:30.738840 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 00:18:30.739139 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":340,"threads_started":1}
I20260811 00:18:30.740614 14569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.723369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:41259 (local address 127.0.0.1:40066)
0811 00:18:30.724206 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.724288 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:30.724448 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:30.724851 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:30.724857 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:30.725873 (+ 1016us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:30.726246 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:30.726259 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.727749 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.727753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:30.729349 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:30.729368 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.730150 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.730160 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:30.730373 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.731007 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:30.731038 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:30.733663 (+ 2625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:30.736565 (+ 2902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:30.736572 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:30.736580 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:30.736949 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:30.737435 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:30.737438 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:30.737440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:30.737589 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:30.738108 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:30.738115 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:30.738340 (+ 225us) client_negotiation.cc:769] Sending connection context
0811 00:18:30.738632 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 00:18:30.739275 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":162,"thread_start_us":431,"threads_started":1}
I20260811 00:18:30.742179 14561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:18:30 dist-test-slave-8hhm krb5kdc[14169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407510, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.60@KRBTEST.COM for kudu/127.7.94.61@KRBTEST.COM
Aug 11 00:18:30 dist-test-slave-8hhm krb5kdc[14169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:18:30.755883 14569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.742805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:46721 (local address 127.0.0.1:52058)
0811 00:18:30.743221 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.743240 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:30.743354 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:30.743738 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:30.743744 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:30.744109 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:30.744234 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:30.744242 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.745259 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.745269 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:30.746161 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:30.746178 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.746893 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.746898 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:30.747106 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.747664 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:30.747681 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:30.750099 (+ 2418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:30.753414 (+ 3315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:30.753420 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:30.753425 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:30.753945 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:30.754341 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:30.754347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:30.754350 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:30.754522 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:30.755110 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:30.755118 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:30.755285 (+ 167us) client_negotiation.cc:769] Sending connection context
0811 00:18:30.755438 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 00:18:30.755650 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150}
I20260811 00:18:30.756170 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.742976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52058 (local address 127.7.94.61:46721)
0811 00:18:30.743142 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:30.743149 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:30.743321 (+ 172us) server_negotiation.cc:407] Connection header received
0811 00:18:30.743501 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:30.743511 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:30.743612 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:30.743713 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:30.744378 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.745059 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.746313 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.746748 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.746885 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.750279 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:30.750284 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:30.750287 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:30.750319 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:30.753272 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.754068 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.754072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.754074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.754223 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.754644 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.754647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.754649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.754982 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:30.755096 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:30.755568 (+ 472us) server_negotiation.cc:299] Negotiation successful
0811 00:18:30.755793 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 00:18:30.757015 14561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:18:30.758797 14568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.743710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40082 (local address 127.7.94.62:41259)
0811 00:18:30.743976 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:30.743981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:30.743996 (+ 15us) server_negotiation.cc:407] Connection header received
0811 00:18:30.744165 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:30.744170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:30.744245 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:30.744337 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:30.745501 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.746154 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.747334 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.747765 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.747892 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.752224 (+ 4332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:30.752229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:30.752232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:30.752260 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:30.755306 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.756345 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.756350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.756353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.756515 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.756971 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.756977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.756980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.757368 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:30.757684 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:30.758425 (+ 741us) server_negotiation.cc:299] Negotiation successful
0811 00:18:30.758602 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
Aug 11 00:18:30 dist-test-slave-8hhm krb5kdc[14169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407510, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.60@KRBTEST.COM for kudu/127.7.94.60@KRBTEST.COM
Aug 11 00:18:30 dist-test-slave-8hhm krb5kdc[14169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.61@KRBTEST.COM
I20260811 00:18:30.770707 14569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.757821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35880)
0811 00:18:30.758071 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.758087 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:30.758207 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:30.758516 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:30.758520 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:30.758878 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:30.759044 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:30.759054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.760178 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.760183 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:30.761536 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:30.761554 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.762206 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.762211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:30.762373 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.762895 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:30.762915 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:30.765307 (+ 2392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:30.768078 (+ 2771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:30.768085 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:30.768092 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:30.768448 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:30.768875 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:30.768882 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:30.768888 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:30.769049 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:30.769561 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:30.769565 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:30.769714 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 00:18:30.769886 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 00:18:30.770181 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260811 00:18:30.770713 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.757791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35880 (local address 127.7.94.60:41571)
0811 00:18:30.757970 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:30.757976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:30.758158 (+ 182us) server_negotiation.cc:407] Connection header received
0811 00:18:30.758343 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:30.758350 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:30.758418 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:30.758496 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:30.759184 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.760020 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.761674 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.762047 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.762141 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.765494 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:30.765498 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:30.765501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:30.765527 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:30.767937 (+ 2410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.768599 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.768604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.768608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.768744 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.769152 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.769158 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.769161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.769446 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:30.769592 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:30.769986 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 00:18:30.770354 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":145,"server-negotiator.queue_time_us":59}
I20260811 00:18:30.775732 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.760318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52060 (local address 127.7.94.61:46721)
0811 00:18:30.760462 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:30.760466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:30.760480 (+ 14us) server_negotiation.cc:407] Connection header received
0811 00:18:30.760626 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:30.760629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:30.760691 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:30.760764 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:30.762255 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.762893 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.764525 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.765050 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.765143 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.769785 (+ 4642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:30.769791 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:30.769795 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:30.769831 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:30.773311 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.773899 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.773904 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.773908 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.774085 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.774438 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.774443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.774446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.774761 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:30.775033 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:30.775192 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 00:18:30.775423 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 00:18:30.776847 14561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98: Bootstrap starting.
I20260811 00:18:30.778230 14302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } attempt: 4
I20260811 00:18:30.781072 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:18:30.781486 14561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:30.782652 14561 log.cc:824] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:30.786523 14561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98: No bootstrap required, opened a new log
Aug 11 00:18:30 dist-test-slave-8hhm krb5kdc[14169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407509, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for kudu/127.7.94.60@KRBTEST.COM
I20260811 00:18:30.784659 14260 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "85e5d0c211f24aae93187fa6adb60f76"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40082
I20260811 00:18:30.795811 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.778880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35882)
0811 00:18:30.779043 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.779063 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:30.779170 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:30.779817 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:30.779823 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:30.780211 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:30.780376 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:30.780384 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.781772 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.781777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:30.783280 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:30.783297 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.784360 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.784365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:30.784581 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.785245 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:30.785271 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:30.788365 (+ 3094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:30.792694 (+ 4329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:30.792699 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:30.792704 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:30.793195 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:30.793625 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:30.793629 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:30.793633 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:30.793956 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:30.794720 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:30.794725 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:30.794909 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 00:18:30.795106 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 00:18:30.795420 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 00:18:30.795825 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.778924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35882 (local address 127.7.94.60:41571)
0811 00:18:30.779423 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:30.779430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:30.779453 (+ 23us) server_negotiation.cc:407] Connection header received
0811 00:18:30.779523 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:30.779528 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:30.779603 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:30.779767 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:30.780515 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.781620 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.783456 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.784205 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.784350 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.788573 (+ 4223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:30.788580 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:30.788597 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:30.788633 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:30.792553 (+ 3920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.793332 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.793339 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.793343 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.793501 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.794132 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.794137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.794140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.794594 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:30.794765 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:30.795230 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 00:18:30.795454 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 00:18:30.796825 14561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e5d0c211f24aae93187fa6adb60f76" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } } peers { permanent_uuid: "145b7c17e33f4ecda830340911cf39f5" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } } peers { permanent_uuid: "193f1bd425994c5bba235db3a97c8a98" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } }
I20260811 00:18:30.797290 14561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193f1bd425994c5bba235db3a97c8a98, State: Initialized, Role: FOLLOWER
I20260811 00:18:30.797876 14561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e5d0c211f24aae93187fa6adb60f76" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } } peers { permanent_uuid: "145b7c17e33f4ecda830340911cf39f5" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } } peers { permanent_uuid: "193f1bd425994c5bba235db3a97c8a98" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } }
I20260811 00:18:30.798935 14571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e5d0c211f24aae93187fa6adb60f76" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } } peers { permanent_uuid: "145b7c17e33f4ecda830340911cf39f5" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } } peers { permanent_uuid: "193f1bd425994c5bba235db3a97c8a98" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } } }
I20260811 00:18:30.799341 14571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:18:30.800237 14561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:18:30.802043 14302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76: Bootstrap starting.
I20260811 00:18:30.806517 14302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:30.807925 14302 log.cc:824] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:30.811198 14302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76: No bootstrap required, opened a new log
I20260811 00:18:30.818897 14302 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e5d0c211f24aae93187fa6adb60f76" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } } peers { permanent_uuid: "145b7c17e33f4ecda830340911cf39f5" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } } peers { permanent_uuid: "193f1bd425994c5bba235db3a97c8a98" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } }
I20260811 00:18:30.819307 14302 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e5d0c211f24aae93187fa6adb60f76, State: Initialized, Role: FOLLOWER
W20260811 00:18:30.819802 14590 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:18:30.819947 14590 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:18:30.819828 14302 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e5d0c211f24aae93187fa6adb60f76" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } } peers { permanent_uuid: "145b7c17e33f4ecda830340911cf39f5" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } } peers { permanent_uuid: "193f1bd425994c5bba235db3a97c8a98" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } }
I20260811 00:18:30.808179 14440 master_runner.cc:186] Error getting master registration for 127.7.94.62:41259: OK, instance_id { permanent_uuid: "85e5d0c211f24aae93187fa6adb60f76" instance_seqno: 1786407509636430 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 00:18:30.821621 14440 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 00:18:30.826817 14596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e5d0c211f24aae93187fa6adb60f76" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } } peers { permanent_uuid: "145b7c17e33f4ecda830340911cf39f5" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } } peers { permanent_uuid: "193f1bd425994c5bba235db3a97c8a98" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } } }
I20260811 00:18:30.827278 14596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:18:30.828694 14302 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:18:30.828783 14260 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 00:18:30.828971 14260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:30.833303 14260 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e5d0c211f24aae93187fa6adb60f76" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } } peers { permanent_uuid: "145b7c17e33f4ecda830340911cf39f5" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } } peers { permanent_uuid: "193f1bd425994c5bba235db3a97c8a98" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } }
I20260811 00:18:30.836813 14260 leader_election.cc:288] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [CANDIDATE]: Term 1 election: Requested vote from peers 145b7c17e33f4ecda830340911cf39f5 (127.7.94.61:46721), 193f1bd425994c5bba235db3a97c8a98 (127.7.94.60:41571)
I20260811 00:18:30.837412 14509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "85e5d0c211f24aae93187fa6adb60f76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "193f1bd425994c5bba235db3a97c8a98"
I20260811 00:18:30.837935 14509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:30.841645 14509 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e5d0c211f24aae93187fa6adb60f76 in term 1.
I20260811 00:18:30.842408 14223 leader_election.cc:302] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193f1bd425994c5bba235db3a97c8a98, 85e5d0c211f24aae93187fa6adb60f76; no voters:
I20260811 00:18:30.842916 14596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 00:18:30.853406 14596 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [term 1 LEADER]: Becoming Leader. State: Replica: 85e5d0c211f24aae93187fa6adb60f76, State: Running, Role: LEADER
I20260811 00:18:30.854081 14596 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e5d0c211f24aae93187fa6adb60f76" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } } peers { permanent_uuid: "145b7c17e33f4ecda830340911cf39f5" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } } peers { permanent_uuid: "193f1bd425994c5bba235db3a97c8a98" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } }
I20260811 00:18:30.861930 14609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85e5d0c211f24aae93187fa6adb60f76. Latest consensus state: current_term: 1 leader_uuid: "85e5d0c211f24aae93187fa6adb60f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e5d0c211f24aae93187fa6adb60f76" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } } peers { permanent_uuid: "145b7c17e33f4ecda830340911cf39f5" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } } peers { permanent_uuid: "193f1bd425994c5bba235db3a97c8a98" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } } }
I20260811 00:18:30.862413 14609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:30.868353 14615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260811 00:18:30.876045 14227 tablet.cc:2369] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:18:30.885381 14615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:18:30.892838 14428 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } attempt: 5
Aug 11 00:18:30 dist-test-slave-8hhm krb5kdc[14169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407510, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.61@KRBTEST.COM for kudu/127.7.94.60@KRBTEST.COM
I20260811 00:18:30.914083 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.893683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35884 (local address 127.7.94.60:41571)
0811 00:18:30.893873 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:30.893879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:30.894157 (+ 278us) server_negotiation.cc:407] Connection header received
0811 00:18:30.894241 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:30.894246 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:30.894364 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:30.894471 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:30.895963 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.897049 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.898884 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.899479 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.899623 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.905393 (+ 5770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:30.905400 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:30.905405 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:30.905445 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:30.909410 (+ 3965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.910738 (+ 1328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.910743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.910746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.910914 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.911559 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.911565 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.911568 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.911944 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:30.912093 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:30.912952 (+ 859us) server_negotiation.cc:299] Negotiation successful
0811 00:18:30.913416 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:18:30.915433 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.893563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:41571 (local address 127.0.0.1:35884)
0811 00:18:30.893870 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:30.893893 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:30.894036 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:30.895037 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:30.895042 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:30.895533 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:30.895764 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:30.895779 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.897242 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.897247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:30.898689 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:30.898708 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.899846 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.899852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:30.900069 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.900893 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:30.900919 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:30.905108 (+ 4189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:30.909696 (+ 4588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:30.909701 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:30.909705 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:30.910504 (+ 799us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:30.911107 (+ 603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:30.911112 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:30.911115 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:30.911373 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:30.912263 (+ 890us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:30.912268 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:30.912492 (+ 224us) client_negotiation.cc:769] Sending connection context
0811 00:18:30.914468 (+ 1976us) client_negotiation.cc:240] Negotiation successful
0811 00:18:30.914891 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169}
I20260811 00:18:30.924084 14509 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98 [term 1 FOLLOWER]: Refusing update from remote peer 85e5d0c211f24aae93187fa6adb60f76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:18:30.925459 14609 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193f1bd425994c5bba235db3a97c8a98" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:18:30.932320 14428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 145b7c17e33f4ecda830340911cf39f5: Bootstrap starting.
W20260811 00:18:30.938970 14470 tablet.cc:2369] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:18:30.944252 14428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 145b7c17e33f4ecda830340911cf39f5: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:30.946762 14571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85e5d0c211f24aae93187fa6adb60f76. Latest consensus state: current_term: 1 leader_uuid: "85e5d0c211f24aae93187fa6adb60f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e5d0c211f24aae93187fa6adb60f76" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } } peers { permanent_uuid: "145b7c17e33f4ecda830340911cf39f5" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } } peers { permanent_uuid: "193f1bd425994c5bba235db3a97c8a98" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } } }
I20260811 00:18:30.947180 14571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:18:30.953317 14568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.921648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40084 (local address 127.7.94.62:41259)
0811 00:18:30.921840 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:30.921846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:30.921863 (+ 17us) server_negotiation.cc:407] Connection header received
0811 00:18:30.921919 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:30.921924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:30.922005 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:30.922099 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:30.925556 (+ 3457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.926722 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.929786 (+ 3064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.930466 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.930575 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.940488 (+ 9913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:30.940495 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:30.940499 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:30.940541 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:30.945587 (+ 5046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.948291 (+ 2704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.948297 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.948301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.948468 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.949733 (+ 1265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.949741 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.949745 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.950169 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:30.950486 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:30.952234 (+ 1748us) server_negotiation.cc:299] Negotiation successful
0811 00:18:30.952630 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 00:18:30.967800 14625 mvcc.cc:205] Tried to move back new op lower bound from 7317125164674035712 to 7317125164464705536. Current Snapshot: MvccSnapshot[applied={T|T < 7317125164674035712}]
I20260811 00:18:30.978892 14596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "85e5d0c211f24aae93187fa6adb60f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e5d0c211f24aae93187fa6adb60f76" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } } peers { permanent_uuid: "145b7c17e33f4ecda830340911cf39f5" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } } peers { permanent_uuid: "193f1bd425994c5bba235db3a97c8a98" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } } }
I20260811 00:18:30.979220 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.922302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35886 (local address 127.7.94.60:41571)
0811 00:18:30.922549 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:30.922557 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:30.922578 (+ 21us) server_negotiation.cc:407] Connection header received
0811 00:18:30.922640 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:30.922647 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:30.922733 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:30.922855 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:30.937651 (+ 14796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.938888 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.953756 (+ 14868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.954524 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.954626 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.966299 (+ 11673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:30.966307 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:30.966318 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:30.966360 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:30.970509 (+ 4149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.973831 (+ 3322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.973837 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.973841 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.974006 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.975262 (+ 1256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.975267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.975271 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.975657 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:30.975918 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:30.978466 (+ 2548us) server_negotiation.cc:299] Negotiation successful
0811 00:18:30.978762 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 00:18:30.979311 14596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:30.983124 14571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "85e5d0c211f24aae93187fa6adb60f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e5d0c211f24aae93187fa6adb60f76" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } } peers { permanent_uuid: "145b7c17e33f4ecda830340911cf39f5" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } } peers { permanent_uuid: "193f1bd425994c5bba235db3a97c8a98" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } } }
I20260811 00:18:30.983464 14571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:18:30.989940 14428 log.cc:824] T 00000000000000000000000000000000 P 145b7c17e33f4ecda830340911cf39f5: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:30.993479 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:30.921922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52070 (local address 127.7.94.61:46721)
0811 00:18:30.922129 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:30.922137 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:30.957672 (+ 35535us) server_negotiation.cc:407] Connection header received
0811 00:18:30.958312 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:30.958318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:30.958405 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:30.958477 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:30.963031 (+ 4554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.964150 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.976134 (+ 11984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:30.976819 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:30.976920 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:30.984911 (+ 7991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:30.984919 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:30.984925 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:30.984967 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:30.988911 (+ 3944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.990372 (+ 1461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.990378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.990382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.990551 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:30.991208 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:30.991214 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:30.991217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:30.991588 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:30.991846 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:30.992739 (+ 893us) server_negotiation.cc:299] Negotiation successful
0811 00:18:30.992985 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 00:18:30.997980 14615 catalog_manager.cc:1347] Generated new cluster ID: 4481147e19894d3aabbe8f4679acb18a
I20260811 00:18:30.998104 14615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:18:31.013731 14428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 145b7c17e33f4ecda830340911cf39f5: No bootstrap required, opened a new log
I20260811 00:18:31.013758 14638 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 00:18:31.024806 14428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 145b7c17e33f4ecda830340911cf39f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e5d0c211f24aae93187fa6adb60f76" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } } peers { permanent_uuid: "145b7c17e33f4ecda830340911cf39f5" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } } peers { permanent_uuid: "193f1bd425994c5bba235db3a97c8a98" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } }
I20260811 00:18:31.031718 14615 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:18:31.034945 14615 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:18:31.036672 14428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 145b7c17e33f4ecda830340911cf39f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145b7c17e33f4ecda830340911cf39f5, State: Initialized, Role: FOLLOWER
I20260811 00:18:31.037379 14428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 145b7c17e33f4ecda830340911cf39f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e5d0c211f24aae93187fa6adb60f76" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } } peers { permanent_uuid: "145b7c17e33f4ecda830340911cf39f5" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } } peers { permanent_uuid: "193f1bd425994c5bba235db3a97c8a98" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } }
I20260811 00:18:31.021250 14191 master_runner.cc:186] Error getting master registration for 127.7.94.61:46721: OK, instance_id { permanent_uuid: "145b7c17e33f4ecda830340911cf39f5" instance_seqno: 1786407510164322 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 00:18:31.043547 14191 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 00:18:31.051988 14428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 145b7c17e33f4ecda830340911cf39f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:18:31.053712 14381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 145b7c17e33f4ecda830340911cf39f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:31.053610 14640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 145b7c17e33f4ecda830340911cf39f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e5d0c211f24aae93187fa6adb60f76" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } } peers { permanent_uuid: "145b7c17e33f4ecda830340911cf39f5" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } } peers { permanent_uuid: "193f1bd425994c5bba235db3a97c8a98" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } } }
I20260811 00:18:31.054098 14640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 145b7c17e33f4ecda830340911cf39f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:18:31.056941 14382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "85e5d0c211f24aae93187fa6adb60f76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "145b7c17e33f4ecda830340911cf39f5"
I20260811 00:18:31.058749 14381 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 145b7c17e33f4ecda830340911cf39f5 [term 1 FOLLOWER]: Refusing update from remote peer 85e5d0c211f24aae93187fa6adb60f76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:18:31.062155 14615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76: Generated new TSK 0
I20260811 00:18:31.063227 14615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:18:31.066108 14382 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 145b7c17e33f4ecda830340911cf39f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85e5d0c211f24aae93187fa6adb60f76 in term 1.
I20260811 00:18:31.068401 14596 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145b7c17e33f4ecda830340911cf39f5" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:18:31.142793 14655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 145b7c17e33f4ecda830340911cf39f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "85e5d0c211f24aae93187fa6adb60f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e5d0c211f24aae93187fa6adb60f76" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } } peers { permanent_uuid: "145b7c17e33f4ecda830340911cf39f5" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } } peers { permanent_uuid: "193f1bd425994c5bba235db3a97c8a98" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } } }
I20260811 00:18:31.143196 14655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 145b7c17e33f4ecda830340911cf39f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:18:31.143879 14640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 145b7c17e33f4ecda830340911cf39f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85e5d0c211f24aae93187fa6adb60f76. Latest consensus state: current_term: 1 leader_uuid: "85e5d0c211f24aae93187fa6adb60f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e5d0c211f24aae93187fa6adb60f76" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } } peers { permanent_uuid: "145b7c17e33f4ecda830340911cf39f5" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } } peers { permanent_uuid: "193f1bd425994c5bba235db3a97c8a98" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } } }
I20260811 00:18:31.144196 14640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 145b7c17e33f4ecda830340911cf39f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:18:31.174767 14666 catalog_manager.cc:1259] Loaded cluster ID: 4481147e19894d3aabbe8f4679acb18a
I20260811 00:18:31.174974 14666 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 145b7c17e33f4ecda830340911cf39f5: loading cluster ID for follower catalog manager: success
I20260811 00:18:31.178769 14666 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 145b7c17e33f4ecda830340911cf39f5: acquiring CA information for follower catalog manager: success
I20260811 00:18:31.181089 14666 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 145b7c17e33f4ecda830340911cf39f5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 00:18:31.191727 7544 mini_kdc.cc:348] Time spent kdestroy: real 0.109s user 0.001s sys 0.029s
W20260811 00:18:31.240379 14568 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:31.227565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40100 (local address 127.7.94.62:41259)
0811 00:18:31.228104 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.228113 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.238138 (+ 10025us) server_negotiation.cc:407] Connection header received
0811 00:18:31.238696 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.238703 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.238790 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.238870 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:31.240108 (+ 1238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40100: BlockingRecv error: recv got EOF from 127.0.0.1:40100 (error 108)
Metrics: {"server-negotiator.queue_time_us":376}
W20260811 00:18:31.243213 14565 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:31.227838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35902 (local address 127.7.94.60:41571)
0811 00:18:31.228258 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.228264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.241383 (+ 13119us) server_negotiation.cc:407] Connection header received
0811 00:18:31.241599 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.241604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.241688 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.241763 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:31.242935 (+ 1172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35902: BlockingRecv error: recv got EOF from 127.0.0.1:35902 (error 108)
Metrics: {"server-negotiator.queue_time_us":296}
W20260811 00:18:31.245476 14436 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:31.241501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52080 (local address 127.7.94.61:46721)
0811 00:18:31.241675 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.241680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.243935 (+ 2255us) server_negotiation.cc:407] Connection header received
0811 00:18:31.244208 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.244213 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.244286 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.244361 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:31.245261 (+ 900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52080: BlockingRecv error: recv got EOF from 127.0.0.1:52080 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 00:18:31.306696 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:31.286962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52088 (local address 127.7.94.61:46721)
0811 00:18:31.287149 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.287155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.298141 (+ 10986us) server_negotiation.cc:407] Connection header received
0811 00:18:31.298773 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.298780 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.299022 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.299163 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:31.299859 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.300970 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.303024 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.303728 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.303833 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:31.304654 (+ 821us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:31.305392 (+ 738us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:31.305527 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:31.306220 (+ 693us) server_negotiation.cc:299] Negotiation successful
0811 00:18:31.306313 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 00:18:31.349562 14565 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:31.340060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51224 (local address 127.7.94.60:41571)
0811 00:18:31.340286 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.340294 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.347104 (+ 6810us) server_negotiation.cc:407] Connection header received
0811 00:18:31.347688 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.347695 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.347855 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.347946 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:31.349281 (+ 1335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51224: BlockingRecv error: recv got EOF from 127.0.0.1:51224 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 00:18:31.396591 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:31.379106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57210 (local address 127.7.94.61:46721)
0811 00:18:31.379309 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.379315 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.389917 (+ 10602us) server_negotiation.cc:407] Connection header received
0811 00:18:31.390414 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.390421 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.390524 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.390590 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:31.391279 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.392444 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.394325 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.395033 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.395124 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:31.395733 (+ 609us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:31.395893 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:31.395955 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:31.396198 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 00:18:31.396289 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 00:18:31.444197 14565 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:31.435133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51240 (local address 127.7.94.60:41571)
0811 00:18:31.435377 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.435384 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.442142 (+ 6758us) server_negotiation.cc:407] Connection header received
0811 00:18:31.442656 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.442662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.442760 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.442858 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:31.443951 (+ 1093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51240: BlockingRecv error: recv got EOF from 127.0.0.1:51240 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 00:18:31.455904 14596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "85e5d0c211f24aae93187fa6adb60f76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85e5d0c211f24aae93187fa6adb60f76" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 41259 } } peers { permanent_uuid: "145b7c17e33f4ecda830340911cf39f5" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 46721 } } peers { permanent_uuid: "193f1bd425994c5bba235db3a97c8a98" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 41571 } } }
I20260811 00:18:31.456279 14596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85e5d0c211f24aae93187fa6adb60f76 [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:31.469734 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:31.462794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57212 (local address 127.7.94.61:46721)
0811 00:18:31.462974 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.462981 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.463552 (+ 571us) server_negotiation.cc:407] Connection header received
0811 00:18:31.463765 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.463771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.463872 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.463979 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:31.464429 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.465610 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.467429 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.468224 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.468365 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:31.468885 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:31.469043 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:31.469146 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:31.469316 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 00:18:31.469404 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 00:18:31.479375 14565 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:31.476687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51242 (local address 127.7.94.60:41571)
0811 00:18:31.476882 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.476890 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.477838 (+ 948us) server_negotiation.cc:407] Connection header received
0811 00:18:31.478024 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.478030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.478124 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.478232 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:31.479143 (+ 911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51242: BlockingRecv error: recv got EOF from 127.0.0.1:51242 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 00:18:31.502633 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:31.495287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57218 (local address 127.7.94.61:46721)
0811 00:18:31.495560 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.495566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.496125 (+ 559us) server_negotiation.cc:407] Connection header received
0811 00:18:31.496416 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.496421 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.496528 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.496607 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:31.497239 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.498566 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.500609 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.501311 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.501421 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:31.501908 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:31.502025 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:31.502103 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:31.502282 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 00:18:31.502357 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
W20260811 00:18:31.517145 14565 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:31.514526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51250 (local address 127.7.94.60:41571)
0811 00:18:31.514824 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.514831 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.515491 (+ 660us) server_negotiation.cc:407] Connection header received
0811 00:18:31.515713 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.515718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.515815 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.515922 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:31.516914 (+ 992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51250: BlockingRecv error: recv got EOF from 127.0.0.1:51250 (error 108)
Metrics: {"server-negotiator.queue_time_us":153}
I20260811 00:18:31.536808 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:31.529969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57234 (local address 127.7.94.61:46721)
0811 00:18:31.530126 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.530131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.530631 (+ 500us) server_negotiation.cc:407] Connection header received
0811 00:18:31.530809 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.530816 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.530913 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.531020 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:31.531429 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.532674 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.534608 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.535319 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.535460 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:31.535918 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:31.536057 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:31.536167 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:31.536326 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 00:18:31.536417 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 00:18:31.546450 14565 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:31.544156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51266 (local address 127.7.94.60:41571)
0811 00:18:31.544342 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.544348 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.544860 (+ 512us) server_negotiation.cc:407] Connection header received
0811 00:18:31.544996 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.545002 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.545102 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.545236 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:31.546162 (+ 926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51266: BlockingRecv error: recv got EOF from 127.0.0.1:51266 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 00:18:31.573335 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:31.565905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57250 (local address 127.7.94.61:46721)
0811 00:18:31.566087 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.566093 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.566609 (+ 516us) server_negotiation.cc:407] Connection header received
0811 00:18:31.566813 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.566819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.566922 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.567044 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:31.567535 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.568949 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.570941 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.571741 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.571889 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:31.572428 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:31.572577 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:31.572650 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:31.572915 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 00:18:31.573018 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 00:18:31.582937 14565 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:31.580614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51272 (local address 127.7.94.60:41571)
0811 00:18:31.580822 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.580828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.581324 (+ 496us) server_negotiation.cc:407] Connection header received
0811 00:18:31.581506 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.581519 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.581642 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.581764 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:31.582651 (+ 887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51272: BlockingRecv error: recv got EOF from 127.0.0.1:51272 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 00:18:31.630720 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:31.623385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57252 (local address 127.7.94.61:46721)
0811 00:18:31.623590 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.623598 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.624159 (+ 561us) server_negotiation.cc:407] Connection header received
0811 00:18:31.624417 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.624424 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.624528 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.624642 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:31.625279 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.626390 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.628591 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.629300 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.629420 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:31.629900 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:31.630032 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:31.630108 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:31.630299 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 00:18:31.630379 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 00:18:31.651587 14565 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:31.645483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51284 (local address 127.7.94.60:41571)
0811 00:18:31.645701 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.645707 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.650060 (+ 4353us) server_negotiation.cc:407] Connection header received
0811 00:18:31.650397 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.650402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.650489 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.650599 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:31.651346 (+ 747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51284: BlockingRecv error: recv got EOF from 127.0.0.1:51284 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 00:18:31.724479 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:31.718979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57264 (local address 127.7.94.61:46721)
0811 00:18:31.719191 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.719197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.719620 (+ 423us) server_negotiation.cc:407] Connection header received
0811 00:18:31.719791 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.719797 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.719888 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.719998 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:31.720381 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.721515 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.722733 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.723326 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.723453 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:31.723784 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:31.723916 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:31.724009 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:31.724110 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 00:18:31.724184 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 00:18:31.732570 14565 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:31.730695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51292 (local address 127.7.94.60:41571)
0811 00:18:31.730911 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.730920 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.731226 (+ 306us) server_negotiation.cc:407] Connection header received
0811 00:18:31.731417 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.731420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.731498 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.731591 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:31.732410 (+ 819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51292: BlockingRecv error: recv got EOF from 127.0.0.1:51292 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 00:18:31.822077 14590 catalog_manager.cc:1259] Loaded cluster ID: 4481147e19894d3aabbe8f4679acb18a
I20260811 00:18:31.822186 14590 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98: loading cluster ID for follower catalog manager: success
I20260811 00:18:31.824958 14590 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98: acquiring CA information for follower catalog manager: success
I20260811 00:18:31.827078 14590 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 193f1bd425994c5bba235db3a97c8a98: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 00:18:31.869153 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:31.864112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57272 (local address 127.7.94.61:46721)
0811 00:18:31.864337 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.864343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.864587 (+ 244us) server_negotiation.cc:407] Connection header received
0811 00:18:31.864803 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.864812 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.864877 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.864965 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:31.865489 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.866189 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.867479 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.867914 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.868065 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:31.868443 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:31.868560 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:31.868656 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:31.868828 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 00:18:31.868898 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:18:31.879901 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:31.874523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51298 (local address 127.7.94.60:41571)
0811 00:18:31.874727 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.874733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.874898 (+ 165us) server_negotiation.cc:407] Connection header received
0811 00:18:31.875105 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.875109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.875265 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.875384 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:31.875776 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.876493 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.877855 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.878342 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.878472 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:31.878917 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:31.879447 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:31.879563 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:31.879625 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 00:18:31.879687 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 00:18:31.893940 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:31.886079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51312 (local address 127.7.94.60:41571)
0811 00:18:31.886267 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.886275 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.887701 (+ 1426us) server_negotiation.cc:407] Connection header received
0811 00:18:31.887960 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.887969 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.888084 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.888175 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:31.888958 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.889891 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.891957 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.892592 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.892723 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:31.893264 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:31.893376 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:31.893475 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:31.893620 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 00:18:31.893693 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 00:18:31.894476 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:31.887405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57282 (local address 127.7.94.61:46721)
0811 00:18:31.887610 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.887615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.888423 (+ 808us) server_negotiation.cc:407] Connection header received
0811 00:18:31.888703 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.888709 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.888777 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.888868 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:31.889533 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.890572 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.892462 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.893107 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.893267 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:31.893709 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:31.893846 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:31.893913 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:31.894119 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 00:18:31.894196 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 00:18:31.895191 14850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:31.885948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41200 (local address 127.7.94.62:41259)
0811 00:18:31.887798 (+ 1850us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:31.887804 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:31.887821 (+ 17us) server_negotiation.cc:407] Connection header received
0811 00:18:31.887946 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:31.887954 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:31.888205 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:31.888394 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:31.889067 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.889983 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.891737 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:31.892346 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:31.892467 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:31.893004 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:31.893501 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:31.893641 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:31.894657 (+ 1016us) server_negotiation.cc:299] Negotiation successful
0811 00:18:31.894743 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1730,"thread_start_us":1619,"threads_started":1}
I20260811 00:18:31.900162 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 14191
I20260811 00:18:31.901268 14291 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:18:32.011994 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 14191
W20260811 00:18:32.040021 14342 connection.cc:535] server connection from 127.0.0.1:52042 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 00:18:32.040637 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 14316
I20260811 00:18:32.041699 14420 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:18:32.150178 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 14316
I20260811 00:18:32.179210 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 14440
I20260811 00:18:32.180352 14550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:18:32.288251 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 14440
2026-08-11T00:18:32Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4780 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 00:18:32 dist-test-slave-8hhm krb5kdc[14867](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 00:18:32 dist-test-slave-8hhm krb5kdc[14867](info): set up 2 sockets
Aug 11 00:18:32 dist-test-slave-8hhm krb5kdc[14867](info): commencing operation
krb5kdc: starting...
W20260811 00:18:33.678511 7544 mini_kdc.cc:121] Time spent starting KDC: real 1.338s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/test-user.keytab.
Aug 11 00:18:33 dist-test-slave-8hhm krb5kdc[14867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T00:18:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T00:18:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.94.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
Entry for principal kudu/127.7.94.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab.
I20260811 00:18:34.018746 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:43477
--webserver_interface=127.7.94.62
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:35373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:43477,127.7.94.61:44375,127.7.94.60:42459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 00:18:34.368709 14889 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 00:18:34.369154 14889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:34.369304 14889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:34.369450 14889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:34.376991 14889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:34.377123 14889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:34.377270 14889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:34.377359 14889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:34.377467 14889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:34.387573 14889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:35373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:43477,127.7.94.61:44375,127.7.94.60:42459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.62.keytab
--principal=kudu/127.7.94.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.62:43477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.94.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:34.388965 14889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:34.390447 14889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:34.402633 14899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:34.402420 14900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:34.402416 14902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:34.403828 14889 server_base.cc:1034] running on GCE node
I20260811 00:18:34.404662 14889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:34.406244 14889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:34.407462 14889 hybrid_clock.cc:648] HybridClock initialized: now 1786407514407379 us; error 55 us; skew 500 ppm
I20260811 00:18:34.407831 14889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:34 dist-test-slave-8hhm krb5kdc[14867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407514, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:34.411278 14889 init.cc:377] Logged in from keytab as kudu/127.7.94.62@KRBTEST.COM (short username kudu)
I20260811 00:18:34.412940 14889 webserver.cc:460] Webserver started at http://127.7.94.62:45139/ using document root <none> and password file <none>
I20260811 00:18:34.413760 14889 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:34.413909 14889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:34.414254 14889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:34.417434 14889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-0/data/instance:
uuid: "a6a303d651a04a6da3b9ff15668a403f"
format_stamp: "Formatted at 2026-08-11 00:18:34 on dist-test-slave-8hhm"
server_key: "a8de9994b0cff2ebe04354505f7882d0"
server_key_iv: "587f5f33eeca91bce12d0a5386637626"
server_key_version: "encryptionkey@0"
I20260811 00:18:34.418258 14889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-0/wal/instance:
uuid: "a6a303d651a04a6da3b9ff15668a403f"
format_stamp: "Formatted at 2026-08-11 00:18:34 on dist-test-slave-8hhm"
server_key: "a8de9994b0cff2ebe04354505f7882d0"
server_key_iv: "587f5f33eeca91bce12d0a5386637626"
server_key_version: "encryptionkey@0"
I20260811 00:18:34.423671 14889 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 00:18:34.427237 14912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:34.428568 14889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 00:18:34.428761 14889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-0/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-0/wal
uuid: "a6a303d651a04a6da3b9ff15668a403f"
format_stamp: "Formatted at 2026-08-11 00:18:34 on dist-test-slave-8hhm"
server_key: "a8de9994b0cff2ebe04354505f7882d0"
server_key_iv: "587f5f33eeca91bce12d0a5386637626"
server_key_version: "encryptionkey@0"
I20260811 00:18:34.428985 14889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:34.442610 14889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:34.447122 14889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:34.447427 14889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:34.466718 14889 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.62:43477
I20260811 00:18:34.466753 15003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.62:43477 every 8 connection(s)
I20260811 00:18:34.468173 14889 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-0/data/info.pb
I20260811 00:18:34.471994 15004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:34.472815 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 14889
I20260811 00:18:34.473006 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-0/wal/instance
I20260811 00:18:34.473659 7544 external_mini_cluster.cc:1428] Setting key 82f4b3be9ae5d8c1ca697e7a7552a8fa
I20260811 00:18:34.482612 15004 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:18:34 dist-test-slave-8hhm krb5kdc[14867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407514, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:18:34.509057 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:34.486939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:43477 (local address 127.0.0.1:42638)
0811 00:18:34.488334 (+ 1395us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:34.488388 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:34.490315 (+ 1927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:34.491248 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:34.491254 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:34.491844 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:34.492193 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:34.492213 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:34.494602 (+ 2389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:34.494607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:34.495689 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:34.495701 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:34.497092 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:34.497097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:34.497278 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:34.497977 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:34.498005 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:34.500202 (+ 2197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:34.504765 (+ 4563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:34.504775 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:34.504786 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:34.505272 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:34.505743 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:34.505746 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:34.505749 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:34.505913 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:34.506967 (+ 1054us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:34.506973 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:34.507304 (+ 331us) client_negotiation.cc:769] Sending connection context
0811 00:18:34.507525 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 00:18:34.507949 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1107,"spinlock_wait_cycles":16128,"thread_start_us":881,"threads_started":1}
I20260811 00:18:34.509122 15007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:34.486939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42638 (local address 127.7.94.62:43477)
0811 00:18:34.487870 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:34.487888 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:34.488531 (+ 643us) server_negotiation.cc:407] Connection header received
0811 00:18:34.490709 (+ 2178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:34.490715 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:34.490986 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:34.491208 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:34.492451 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:34.494459 (+ 2008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:34.495839 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:34.496879 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:34.497068 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:34.500446 (+ 3378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:34.500459 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:34.500479 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:34.500526 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:34.504635 (+ 4109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:34.505406 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:34.505418 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:34.505424 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:34.505629 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:34.506041 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:34.506049 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:34.506055 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:34.506823 (+ 768us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:34.507233 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:34.507726 (+ 493us) server_negotiation.cc:299] Negotiation successful
0811 00:18:34.508185 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":646,"thread_start_us":389,"threads_started":1}
WARNING: no policy specified for kudu/127.7.94.61@KRBTEST.COM; defaulting to no policy
I20260811 00:18:34.512154 15004 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:18:34.513219 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:34.512769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:44375 (local address 127.0.0.1:52730)
0811 00:18:34.512960 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:34.513078 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260811 00:18:34.513497 14919 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.94.61:44375: connect: Connection refused (error 111)
I20260811 00:18:34.515069 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:34.514712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:44375 (local address 127.0.0.1:52740)
0811 00:18:34.514837 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:34.514902 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.61@KRBTEST.COM" created.
W20260811 00:18:34.515358 15004 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.61:44375: Network error: Client connection negotiation failed: client connection to 127.7.94.61:44375: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.61.keytab.
Entry for principal kudu/127.7.94.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.61.keytab.
I20260811 00:18:34.556802 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.61:44375
--webserver_interface=127.7.94.61
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.61.keytab
--principal=kudu/127.7.94.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:35373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:43477,127.7.94.61:44375,127.7.94.60:42459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:18:34.564705 15004 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } attempt: 1
I20260811 00:18:34.565937 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:34.565485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:44375 (local address 127.0.0.1:52742)
0811 00:18:34.565683 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:34.565797 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 00:18:34.566987 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:34.566654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:44375 (local address 127.0.0.1:52746)
0811 00:18:34.566793 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:34.566859 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 00:18:34.567243 15004 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.61:44375: Network error: Client connection negotiation failed: client connection to 127.7.94.61:44375: connect: Connection refused (error 111)
I20260811 00:18:34.635471 15004 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } attempt: 2
I20260811 00:18:34.636757 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:34.636274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:44375 (local address 127.0.0.1:52754)
0811 00:18:34.636512 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:34.636604 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 00:18:34.637821 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:34.637476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:44375 (local address 127.0.0.1:52766)
0811 00:18:34.637595 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:34.637677 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 00:18:34.638084 15004 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.61:44375: Network error: Client connection negotiation failed: client connection to 127.7.94.61:44375: connect: Connection refused (error 111)
I20260811 00:18:34.729387 15004 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } attempt: 3
I20260811 00:18:34.730696 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:34.730242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:44375 (local address 127.0.0.1:52782)
0811 00:18:34.730433 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:34.730523 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 00:18:34.731763 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:34.731421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:44375 (local address 127.0.0.1:52784)
0811 00:18:34.731544 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:34.731616 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 00:18:34.732043 15004 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.61:44375: Network error: Client connection negotiation failed: client connection to 127.7.94.61:44375: connect: Connection refused (error 111)
I20260811 00:18:34.875375 15004 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } attempt: 4
I20260811 00:18:34.876669 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:34.876148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:44375 (local address 127.0.0.1:52800)
0811 00:18:34.876456 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:34.876544 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
I20260811 00:18:34.877700 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:34.877365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:44375 (local address 127.0.0.1:52814)
0811 00:18:34.877484 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:34.877557 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.61:44375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 00:18:34.877884 15004 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.61:44375: Network error: Client connection negotiation failed: client connection to 127.7.94.61:44375: connect: Connection refused (error 111)
W20260811 00:18:34.908595 15016 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 00:18:34.908931 15016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:34.909013 15016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:34.909101 15016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:34.916426 15016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:34.916519 15016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:34.916576 15016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:34.916620 15016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:34.916673 15016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:34.926626 15016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:35373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:43477,127.7.94.61:44375,127.7.94.60:42459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.61.keytab
--principal=kudu/127.7.94.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.61:44375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.94.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:34.927888 15016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:34.929174 15016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:34.941187 15026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:34.941411 15024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:34.941167 15030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:34.942051 15016 server_base.cc:1034] running on GCE node
I20260811 00:18:34.942838 15016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:34.944167 15016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:34.945335 15016 hybrid_clock.cc:648] HybridClock initialized: now 1786407514945263 us; error 36 us; skew 500 ppm
I20260811 00:18:34.945662 15016 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:34 dist-test-slave-8hhm krb5kdc[14867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407514, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:34.952075 15016 init.cc:377] Logged in from keytab as kudu/127.7.94.61@KRBTEST.COM (short username kudu)
I20260811 00:18:34.953612 15016 webserver.cc:460] Webserver started at http://127.7.94.61:39517/ using document root <none> and password file <none>
I20260811 00:18:34.954284 15016 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:34.954372 15016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:34.954639 15016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:34.957625 15016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-1/data/instance:
uuid: "bf80e0c67a884972ad25ce38fa67312f"
format_stamp: "Formatted at 2026-08-11 00:18:34 on dist-test-slave-8hhm"
server_key: "14be9f93f541e9830c181f6a555cfa4f"
server_key_iv: "151f8686749b7836c3b2d58a3883d2e4"
server_key_version: "encryptionkey@0"
I20260811 00:18:34.958387 15016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-1/wal/instance:
uuid: "bf80e0c67a884972ad25ce38fa67312f"
format_stamp: "Formatted at 2026-08-11 00:18:34 on dist-test-slave-8hhm"
server_key: "14be9f93f541e9830c181f6a555cfa4f"
server_key_iv: "151f8686749b7836c3b2d58a3883d2e4"
server_key_version: "encryptionkey@0"
I20260811 00:18:34.963426 15016 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 00:18:34.966763 15041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:34.968065 15016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 00:18:34.968242 15016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-1/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-1/wal
uuid: "bf80e0c67a884972ad25ce38fa67312f"
format_stamp: "Formatted at 2026-08-11 00:18:34 on dist-test-slave-8hhm"
server_key: "14be9f93f541e9830c181f6a555cfa4f"
server_key_iv: "151f8686749b7836c3b2d58a3883d2e4"
server_key_version: "encryptionkey@0"
I20260811 00:18:34.968452 15016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:34.990860 15016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:34.994863 15016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:34.995143 15016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:35.014221 15016 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.61:44375
I20260811 00:18:35.014240 15129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.61:44375 every 8 connection(s)
I20260811 00:18:35.015959 15016 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-1/data/info.pb
I20260811 00:18:35.020022 15131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:35.021325 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 15016
I20260811 00:18:35.021538 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-1/wal/instance
I20260811 00:18:35.022117 7544 external_mini_cluster.cc:1428] Setting key 3e94b5b9df6bc3a9263235407f76d065
I20260811 00:18:35.030092 15131 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 00:18:35 dist-test-slave-8hhm krb5kdc[14867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407514, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.61@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:18:35.052565 15134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.033493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42654 (local address 127.7.94.62:43477)
0811 00:18:35.034163 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:35.034171 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:35.035203 (+ 1032us) server_negotiation.cc:407] Connection header received
0811 00:18:35.037135 (+ 1932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:35.037143 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:35.037266 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:35.037400 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:35.038884 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.039626 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.041949 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.042357 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.042453 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.046575 (+ 4122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:35.046580 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:35.046583 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:35.046619 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:35.049411 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.050212 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.050217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.050219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.050337 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.050773 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.050778 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.050782 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.051073 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:35.051248 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:35.051964 (+ 716us) server_negotiation.cc:299] Negotiation successful
0811 00:18:35.052192 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":308,"threads_started":1}
I20260811 00:18:35.053594 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.034162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:43477 (local address 127.0.0.1:42654)
0811 00:18:35.035004 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.035050 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:35.036870 (+ 1820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:35.037602 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:35.037610 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:35.038136 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:35.038685 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:35.038703 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.039779 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.039787 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:35.041792 (+ 2005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:35.041818 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.042616 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.042630 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:35.042955 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.043683 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:35.043716 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:35.046361 (+ 2645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:35.049547 (+ 3186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:35.049556 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:35.049571 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:35.050071 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:35.050447 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:35.050452 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:35.050456 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:35.050640 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:35.051222 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:35.051230 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:35.051586 (+ 356us) client_negotiation.cc:769] Sending connection context
0811 00:18:35.051866 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 00:18:35.052368 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":368,"threads_started":1}
I20260811 00:18:35.055289 15131 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.94.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.94.60@KRBTEST.COM" created.
Aug 11 00:18:35 dist-test-slave-8hhm krb5kdc[14867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407514, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.61@KRBTEST.COM for kudu/127.7.94.61@KRBTEST.COM
I20260811 00:18:35.069578 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.055924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:44375 (local address 127.0.0.1:52828)
0811 00:18:35.056143 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.056160 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:35.056241 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:35.057323 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:35.057327 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:35.057864 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:35.058027 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:35.058036 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.059492 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.059497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:35.060329 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:35.060340 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.061147 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.061153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:35.061414 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.061978 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:35.061995 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:35.063926 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:35.067123 (+ 3197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:35.067128 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:35.067132 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:35.067482 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:35.067853 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:35.067858 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:35.067862 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:35.068004 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:35.068568 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:35.068573 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:35.068724 (+ 151us) client_negotiation.cc:769] Sending connection context
0811 00:18:35.068874 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 00:18:35.069151 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":5}
I20260811 00:18:35.069635 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.056223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52828 (local address 127.7.94.61:44375)
0811 00:18:35.056905 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:35.056911 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:35.056939 (+ 28us) server_negotiation.cc:407] Connection header received
0811 00:18:35.056981 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:35.056986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:35.057161 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:35.057334 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:35.058208 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.059342 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.060461 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.061005 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.061140 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.064123 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:35.064133 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:35.064145 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:35.064173 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:35.066988 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.067608 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.067620 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.067622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.067745 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.068102 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.068110 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.068113 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.068445 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:35.068673 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:35.068996 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 00:18:35.069270 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":97,"server-negotiator.queue_time_us":492,"thread_start_us":365,"threads_started":1}
I20260811 00:18:35.072336 15131 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:18:35.073575 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.072952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40756)
0811 00:18:35.073183 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.073358 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 00:18:35.073869 15045 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
I20260811 00:18:35.075485 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.075073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40760)
0811 00:18:35.075257 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.075359 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 00:18:35.075765 15131 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:42459: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.94.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.60.keytab.
Entry for principal kudu/127.7.94.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.60.keytab.
I20260811 00:18:35.100687 7544 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1KEo_S/build/asan/bin/kudu
/tmp/dist-test-task1KEo_S/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.60:42459
--webserver_interface=127.7.94.60
--webserver_port=0
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.60.keytab
--principal=kudu/127.7.94.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.94.20:35373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.94.62:43477,127.7.94.61:44375,127.7.94.60:42459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 00:18:35.125082 15131 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } attempt: 1
I20260811 00:18:35.126473 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.125855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40772)
0811 00:18:35.126118 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.126252 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 00:18:35.127645 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.127201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40776)
0811 00:18:35.127396 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.127496 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 00:18:35.127864 15131 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:42459: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
I20260811 00:18:35.177170 15004 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } attempt: 5
Aug 11 00:18:35 dist-test-slave-8hhm krb5kdc[14867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407514, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for kudu/127.7.94.61@KRBTEST.COM
I20260811 00:18:35.191318 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.177994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:44375 (local address 127.0.0.1:52830)
0811 00:18:35.178297 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.178319 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:35.178461 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:35.178837 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:35.178842 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:35.179268 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:35.179440 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:35.179449 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.180441 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.180451 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:35.181405 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:35.181418 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.182132 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.182137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:35.182326 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.182982 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:35.183000 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:35.185289 (+ 2289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:35.188935 (+ 3646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:35.188939 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:35.188941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:35.189380 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:35.189790 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:35.189793 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:35.189795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:35.189920 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:35.190518 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:35.190521 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:35.190654 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 00:18:35.190785 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 00:18:35.191040 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 00:18:35.191464 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.178056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52830 (local address 127.7.94.61:44375)
0811 00:18:35.178368 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:35.178375 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:35.178397 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:18:35.178620 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:35.178625 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:35.178692 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:35.178800 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:35.179590 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.180295 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.181558 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.181991 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.182124 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.185474 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:35.185480 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:35.185485 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:35.185525 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:35.188811 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.189496 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.189503 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.189509 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.189677 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.190030 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.190036 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.190039 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.190404 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:35.190541 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:35.190910 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 00:18:35.191138 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191}
I20260811 00:18:35.192433 15004 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:18:35.193428 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.193012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40792)
0811 00:18:35.193164 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.193275 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 00:18:35.194363 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.193975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40802)
0811 00:18:35.194117 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.194221 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 00:18:35.194586 15004 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:42459: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
I20260811 00:18:35.196091 15131 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } attempt: 2
I20260811 00:18:35.197134 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.196628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40818)
0811 00:18:35.196833 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.196940 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 00:18:35.198251 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.197839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40832)
0811 00:18:35.197998 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.198086 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 00:18:35.198473 15131 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:42459: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
I20260811 00:18:35.245857 15004 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } attempt: 1
I20260811 00:18:35.247187 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.246615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40846)
0811 00:18:35.246825 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.246961 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 00:18:35.248231 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.247852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40858)
0811 00:18:35.248002 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.248090 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 00:18:35.248452 15004 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:42459: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
I20260811 00:18:35.289688 15131 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } attempt: 3
I20260811 00:18:35.290916 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.290353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40864)
0811 00:18:35.290586 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.290706 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 00:18:35.292074 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.291590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40872)
0811 00:18:35.291834 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.291929 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
W20260811 00:18:35.292284 15131 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:42459: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
I20260811 00:18:35.316705 15004 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } attempt: 2
I20260811 00:18:35.318228 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.317506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40882)
0811 00:18:35.317802 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.317985 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 00:18:35.319550 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.319046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40884)
0811 00:18:35.319236 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.319349 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 00:18:35.319847 15004 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:42459: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
I20260811 00:18:35.426087 15004 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } attempt: 3
I20260811 00:18:35.427388 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.426723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40892)
0811 00:18:35.426996 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.427144 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 00:18:35.428589 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.428142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40898)
0811 00:18:35.428330 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.428428 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 00:18:35.428871 15004 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:42459: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
I20260811 00:18:35.435520 15131 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } attempt: 4
I20260811 00:18:35.436735 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.436139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40914)
0811 00:18:35.436392 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.436516 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 00:18:35.437872 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.437490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40918)
0811 00:18:35.437650 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.437742 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 00:18:35.438133 15131 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.94.60:42459: Network error: Client connection negotiation failed: client connection to 127.7.94.60:42459: connect: Connection refused (error 111)
W20260811 00:18:35.486380 15141 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 00:18:35.486681 15141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 00:18:35.486749 15141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 00:18:35.486824 15141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 00:18:35.493957 15141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 00:18:35.494042 15141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 00:18:35.494098 15141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 00:18:35.494148 15141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 00:18:35.494208 15141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 00:18:35.503566 15141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.94.20:35373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.94.62:43477,127.7.94.61:44375,127.7.94.60:42459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/krb5kdc/kudu_127.7.94.60.keytab
--principal=kudu/127.7.94.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.94.60:42459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.94.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 22:46:59 UTC on ccycloud.quasar-yjrvid.root.comops.site
build id 4579238
ASAN enabled
I20260811 00:18:35.504714 15141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 00:18:35.506016 15141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 00:18:35.518035 15155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:35.518173 15157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 00:18:35.518183 15154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 00:18:35.518949 15141 server_base.cc:1034] running on GCE node
I20260811 00:18:35.519794 15141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 00:18:35.521193 15141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 00:18:35.522482 15141 hybrid_clock.cc:648] HybridClock initialized: now 1786407515522407 us; error 46 us; skew 500 ppm
I20260811 00:18:35.522830 15141 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 00:18:35 dist-test-slave-8hhm krb5kdc[14867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407515, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 00:18:35.526027 15141 init.cc:377] Logged in from keytab as kudu/127.7.94.60@KRBTEST.COM (short username kudu)
I20260811 00:18:35.527469 15141 webserver.cc:460] Webserver started at http://127.7.94.60:33077/ using document root <none> and password file <none>
I20260811 00:18:35.528180 15141 fs_manager.cc:359] Metadata directory not provided
I20260811 00:18:35.528309 15141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 00:18:35.528636 15141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 00:18:35.532085 15141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-2/data/instance:
uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c"
format_stamp: "Formatted at 2026-08-11 00:18:35 on dist-test-slave-8hhm"
server_key: "a067c8c7935028e634fc5a92fe8a7ff9"
server_key_iv: "c913a05d8df5b31397959afae10dfc4b"
server_key_version: "encryptionkey@0"
I20260811 00:18:35.533030 15141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-2/wal/instance:
uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c"
format_stamp: "Formatted at 2026-08-11 00:18:35 on dist-test-slave-8hhm"
server_key: "a067c8c7935028e634fc5a92fe8a7ff9"
server_key_iv: "c913a05d8df5b31397959afae10dfc4b"
server_key_version: "encryptionkey@0"
I20260811 00:18:35.538522 15141 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 00:18:35.541949 15166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 00:18:35.543322 15141 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 00:18:35.543493 15141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-2/data,/tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-2/wal
uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c"
format_stamp: "Formatted at 2026-08-11 00:18:35 on dist-test-slave-8hhm"
server_key: "a067c8c7935028e634fc5a92fe8a7ff9"
server_key_iv: "c913a05d8df5b31397959afae10dfc4b"
server_key_version: "encryptionkey@0"
I20260811 00:18:35.543704 15141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 00:18:35.562296 15141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 00:18:35.566643 15141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 00:18:35.566923 15141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 00:18:35.584815 15141 rpc_server.cc:307] RPC server started. Bound to: 127.7.94.60:42459
I20260811 00:18:35.584833 15263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.94.60:42459 every 8 connection(s)
I20260811 00:18:35.586295 15141 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-2/data/info.pb
I20260811 00:18:35.590005 15265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 00:18:35.593086 7544 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1KEo_S/build/asan/bin/kudu as pid 15141
I20260811 00:18:35.593299 7544 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1KEo_S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786407133955383-7544-0/minicluster-data/master-2/wal/instance
I20260811 00:18:35.594045 7544 external_mini_cluster.cc:1428] Setting key 8a4de2edb97a02cc1ed670b8d4a055d3
I20260811 00:18:35.598278 15265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:18:35.606098 15004 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } attempt: 4
Aug 11 00:18:35 dist-test-slave-8hhm krb5kdc[14867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.60@KRBTEST.COM
Aug 11 00:18:35 dist-test-slave-8hhm krb5kdc[14867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407515, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.60@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
Aug 11 00:18:35 dist-test-slave-8hhm krb5kdc[14867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407514, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.62@KRBTEST.COM for kudu/127.7.94.60@KRBTEST.COM
I20260811 00:18:35.619475 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.601514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42670 (local address 127.7.94.62:43477)
0811 00:18:35.602090 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:35.602096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:35.602614 (+ 518us) server_negotiation.cc:407] Connection header received
0811 00:18:35.602777 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:35.602781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:35.602854 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:35.602944 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:35.604194 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.605104 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.607394 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.607865 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.607979 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.612253 (+ 4274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:35.612258 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:35.612261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:35.612289 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:35.615848 (+ 3559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.616742 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.616750 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.616754 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.616901 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.617463 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.617468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.617471 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.617811 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:35.618242 (+ 431us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:35.618671 (+ 429us) server_negotiation.cc:299] Negotiation successful
0811 00:18:35.618880 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":254,"threads_started":1}
I20260811 00:18:35.620512 15269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.595959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40924 (local address 127.7.94.60:42459)
0811 00:18:35.596758 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:35.596769 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:35.596809 (+ 40us) server_negotiation.cc:407] Connection header received
0811 00:18:35.598608 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:35.598632 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:35.599031 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:35.599378 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:35.601268 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.603380 (+ 2112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.604857 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.605837 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.606022 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.611560 (+ 5538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:35.611570 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:35.611591 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:35.611645 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:35.615508 (+ 3863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.616338 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.616347 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.616351 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.616594 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.617050 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.617057 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.617061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.617986 (+ 925us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:35.619265 (+ 1279us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:35.619460 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 00:18:35.619925 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"spinlock_wait_cycles":18688,"thread_start_us":368,"threads_started":1}
I20260811 00:18:35.620568 15272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.601558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.62:43477 (local address 127.0.0.1:42670)
0811 00:18:35.602450 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.602507 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:35.602660 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:35.603045 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:35.603052 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:35.603806 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:35.604046 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:35.604055 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.605279 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.605284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:35.607206 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:35.607218 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.608243 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.608252 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:35.608469 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.609327 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:35.609361 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:35.612069 (+ 2708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:35.615982 (+ 3913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:35.615993 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:35.616007 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:35.616594 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:35.617009 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:35.617019 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:35.617021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:35.617316 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:35.617984 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:35.617991 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:35.618248 (+ 257us) client_negotiation.cc:769] Sending connection context
0811 00:18:35.618559 (+ 311us) client_negotiation.cc:240] Negotiation successful
0811 00:18:35.619122 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":463,"threads_started":1}
I20260811 00:18:35.622161 15265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:18:35.622509 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.606734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40934)
0811 00:18:35.606886 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.606904 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:35.606998 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:35.607873 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:35.607877 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:35.608323 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:35.608455 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:35.608463 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.610156 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.610161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:35.611327 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:35.611345 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.612434 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.612441 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:35.612612 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.613135 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:35.613160 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:35.616105 (+ 2945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:35.619823 (+ 3718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:35.619830 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:35.619836 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:35.620311 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:35.620797 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:35.620804 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:35.620807 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:35.620968 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:35.621603 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:35.621609 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:35.621741 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 00:18:35.621883 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 00:18:35.622149 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 00:18:35.623592 15273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.606916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40934 (local address 127.7.94.60:42459)
0811 00:18:35.607537 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:35.607543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:35.607565 (+ 22us) server_negotiation.cc:407] Connection header received
0811 00:18:35.607627 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:35.607632 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:35.607695 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:35.607815 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:35.608617 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.610001 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.611505 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.612225 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.612404 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.616305 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:35.616310 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:35.616313 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:35.616340 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:35.619674 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.620483 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.620490 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.620503 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.620658 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.621118 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.621126 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.621129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.621437 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:35.622877 (+ 1440us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:35.622996 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 00:18:35.623224 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":1090,"threads_started":1}
I20260811 00:18:35.628237 15004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f: Bootstrap starting.
Aug 11 00:18:35 dist-test-slave-8hhm krb5kdc[14867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407515, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.60@KRBTEST.COM for kudu/127.7.94.61@KRBTEST.COM
I20260811 00:18:35.632577 15004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:35.634004 15004 log.cc:824] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:35.636245 15272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.622767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.61:44375 (local address 127.0.0.1:52832)
0811 00:18:35.623036 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.623054 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:35.623150 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:35.623924 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:35.623928 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:35.624395 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:35.624638 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:35.624653 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.625653 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.625659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:35.626501 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:35.626512 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.627177 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.627183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:35.627367 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.627887 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:35.627903 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:35.629862 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:35.633651 (+ 3789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:35.633656 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:35.633659 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:35.634367 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:35.634738 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:35.634743 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:35.634746 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:35.634884 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:35.635374 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:35.635378 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:35.635525 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 00:18:35.635655 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 00:18:35.635908 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128}
I20260811 00:18:35.636673 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.623149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52832 (local address 127.7.94.61:44375)
0811 00:18:35.623347 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:35.623353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:35.623373 (+ 20us) server_negotiation.cc:407] Connection header received
0811 00:18:35.623481 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:35.623486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:35.623576 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:35.623654 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:35.624774 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.625512 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.626639 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.627044 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.627170 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.630061 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:35.630069 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:35.630075 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:35.630133 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:35.633501 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.634477 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.634484 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.634490 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.634629 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.634996 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.635000 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.635004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.635259 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:35.635403 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:35.635744 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 00:18:35.635938 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 00:18:35 dist-test-slave-8hhm krb5kdc[14867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.62@KRBTEST.COM
I20260811 00:18:35.637693 15265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } has no permanent_uuid. Determining permanent_uuid...
I20260811 00:18:35.643589 15004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f: No bootstrap required, opened a new log
Aug 11 00:18:35 dist-test-slave-8hhm krb5kdc[14867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407515, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.60@KRBTEST.COM for kudu/127.7.94.60@KRBTEST.COM
I20260811 00:18:35.645869 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.623842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42682 (local address 127.7.94.62:43477)
0811 00:18:35.623992 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:35.623999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:35.624298 (+ 299us) server_negotiation.cc:407] Connection header received
0811 00:18:35.624472 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:35.624476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:35.624563 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:35.624683 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:35.629960 (+ 5277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.630707 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.632471 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.633254 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.633389 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.638846 (+ 5457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:35.638855 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:35.638862 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:35.638912 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:35.642961 (+ 4049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.643569 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.643578 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.643582 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.643721 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.644161 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.644167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.644170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.644535 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:35.644802 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:35.645266 (+ 464us) server_negotiation.cc:299] Negotiation successful
0811 00:18:35.645524 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 00:18:35.652657 15272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.638508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40946)
0811 00:18:35.638924 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.638937 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:35.639059 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:35.639354 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:35.639358 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:35.639695 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:35.639884 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:35.639899 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.640814 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.640821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:35.641842 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:35.641854 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.642516 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.642521 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:35.642701 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.643629 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:35.643644 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:35.646049 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:35.649944 (+ 3895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:35.649949 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:35.649954 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:35.650467 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:35.650865 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:35.650870 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:35.650875 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:35.651032 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:35.651562 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:35.651566 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:35.651706 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 00:18:35.651842 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 00:18:35.652189 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":25}
I20260811 00:18:35.652688 15273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.638726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40946 (local address 127.7.94.60:42459)
0811 00:18:35.638864 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:35.638867 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:35.639006 (+ 139us) server_negotiation.cc:407] Connection header received
0811 00:18:35.639169 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:35.639173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:35.639247 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:35.639336 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:35.640052 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.640691 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.641991 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.642371 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.642511 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.646226 (+ 3715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:35.646230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:35.646233 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:35.646275 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:35.649799 (+ 3524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.650589 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.650596 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.650599 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.650745 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.651134 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.651140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.651143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.651421 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:35.651594 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:35.651927 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 00:18:35.652343 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":165,"server-negotiator.queue_time_us":53}
I20260811 00:18:35.654500 15004 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a303d651a04a6da3b9ff15668a403f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } } peers { permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } } peers { permanent_uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } }
I20260811 00:18:35.654946 15004 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6a303d651a04a6da3b9ff15668a403f, State: Initialized, Role: FOLLOWER
I20260811 00:18:35.655520 15004 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a303d651a04a6da3b9ff15668a403f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } } peers { permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } } peers { permanent_uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } }
I20260811 00:18:35.656607 15277 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a303d651a04a6da3b9ff15668a403f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } } peers { permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } } peers { permanent_uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } } }
I20260811 00:18:35.657119 15277 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [sys.catalog]: This master's current role is: FOLLOWER
Aug 11 00:18:35 dist-test-slave-8hhm krb5kdc[14867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.94.61@KRBTEST.COM
I20260811 00:18:35.658167 15265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c: Bootstrap starting.
I20260811 00:18:35.658260 15004 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:18:35.662509 15265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:35.663952 15265 log.cc:824] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:35.666723 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.647780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52838 (local address 127.7.94.61:44375)
0811 00:18:35.647984 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:35.647989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:35.648006 (+ 17us) server_negotiation.cc:407] Connection header received
0811 00:18:35.648386 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:35.648393 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:35.648467 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:35.648576 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:35.649776 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.650845 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.652593 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.653331 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.653453 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.659163 (+ 5710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:35.659188 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:35.659192 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:35.659234 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:35.663111 (+ 3877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.664043 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.664048 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.664051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.664215 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.664709 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.664718 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.664723 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.665203 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:35.665620 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:35.665915 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 00:18:35.666223 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 00:18:35.667207 15265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c: No bootstrap required, opened a new log
I20260811 00:18:35.672703 7544 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 00:18:35.676043 14954 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a6a303d651a04a6da3b9ff15668a403f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42682
I20260811 00:18:35.676504 14954 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 00:18:35.676733 14954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:35.677659 15265 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a303d651a04a6da3b9ff15668a403f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } } peers { permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } } peers { permanent_uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } }
I20260811 00:18:35.678134 15265 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c44dd0c8b64b10b9a2d74a84294e4c, State: Initialized, Role: FOLLOWER
I20260811 00:18:35.678660 15265 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a303d651a04a6da3b9ff15668a403f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } } peers { permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } } peers { permanent_uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } }
I20260811 00:18:35.680548 14954 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a303d651a04a6da3b9ff15668a403f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } } peers { permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } } peers { permanent_uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } }
W20260811 00:18:35.681133 15298 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 00:18:35.681389 15298 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:18:35.685603 15300 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a303d651a04a6da3b9ff15668a403f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } } peers { permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } } peers { permanent_uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } } }
I20260811 00:18:35.686127 15300 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:18:35.686774 14954 leader_election.cc:288] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [CANDIDATE]: Term 1 election: Requested vote from peers bf80e0c67a884972ad25ce38fa67312f (127.7.94.61:44375), e3c44dd0c8b64b10b9a2d74a84294e4c (127.7.94.60:42459)
I20260811 00:18:35.687014 15265 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:18:35.688453 15214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a6a303d651a04a6da3b9ff15668a403f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c"
I20260811 00:18:35.689042 15214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:35.693707 15214 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6a303d651a04a6da3b9ff15668a403f in term 1.
I20260811 00:18:35.694485 14919 leader_election.cc:302] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6a303d651a04a6da3b9ff15668a403f, e3c44dd0c8b64b10b9a2d74a84294e4c; no voters:
I20260811 00:18:35.695037 15277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [term 1 FOLLOWER]: Leader election won for term 1
W20260811 00:18:35.700443 14924 tablet.cc:2369] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 00:18:35.685822 14889 master_runner.cc:186] Error getting master registration for 127.7.94.61:44375: OK, instance_id { permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" instance_seqno: 1786407515008675 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 00:18:35.709970 14889 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 00:18:35.715406 15277 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [term 1 LEADER]: Becoming Leader. State: Replica: a6a303d651a04a6da3b9ff15668a403f, State: Running, Role: LEADER
I20260811 00:18:35.716094 15277 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a303d651a04a6da3b9ff15668a403f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } } peers { permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } } peers { permanent_uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } }
I20260811 00:18:35.728528 15307 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6a303d651a04a6da3b9ff15668a403f. Latest consensus state: current_term: 1 leader_uuid: "a6a303d651a04a6da3b9ff15668a403f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a303d651a04a6da3b9ff15668a403f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } } peers { permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } } peers { permanent_uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } } }
I20260811 00:18:35.728957 15307 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:35.737346 15131 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } attempt: 5
I20260811 00:18:35.737644 15317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 11 00:18:35 dist-test-slave-8hhm krb5kdc[14867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786407514, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.94.61@KRBTEST.COM for kudu/127.7.94.60@KRBTEST.COM
I20260811 00:18:35.755275 15317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 00:18:35.759794 15270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.716460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42698 (local address 127.7.94.62:43477)
0811 00:18:35.716733 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:35.716742 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:35.718151 (+ 1409us) server_negotiation.cc:407] Connection header received
0811 00:18:35.718738 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:35.718748 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:35.718856 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:35.718945 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:35.725665 (+ 6720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.727247 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.732230 (+ 4983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.735141 (+ 2911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.735302 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.744917 (+ 9615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:35.744928 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:35.744935 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:35.745013 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:35.751135 (+ 6122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.753163 (+ 2028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.753173 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.753179 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.753651 (+ 472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.754738 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.754773 (+ 35us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.754778 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.755896 (+ 1118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:35.756609 (+ 713us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:35.758040 (+ 1431us) server_negotiation.cc:299] Negotiation successful
0811 00:18:35.758699 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 00:18:35.763599 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.717930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52846 (local address 127.7.94.61:44375)
0811 00:18:35.718143 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:35.718149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:35.725019 (+ 6870us) server_negotiation.cc:407] Connection header received
0811 00:18:35.726386 (+ 1367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:35.726392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:35.726497 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:35.726572 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:35.731150 (+ 4578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.732302 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.739229 (+ 6927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.739935 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.740041 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.746589 (+ 6548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:35.746601 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:35.746607 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:35.746645 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:35.751364 (+ 4719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.760081 (+ 8717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.760086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.760089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.760247 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.761686 (+ 1439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.761694 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.761698 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.762087 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:35.762338 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:35.762914 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 00:18:35.763210 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 00:18:35.765781 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.738143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.94.60:42459 (local address 127.0.0.1:40960)
0811 00:18:35.738648 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 00:18:35.738674 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 00:18:35.738802 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 00:18:35.739673 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 00:18:35.739678 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 00:18:35.740209 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0811 00:18:35.740404 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:35.740416 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.741943 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.741949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:35.743524 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 00:18:35.743539 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.744739 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.744746 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 00:18:35.744963 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.745625 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 00:18:35.745650 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 00:18:35.751597 (+ 5947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 00:18:35.758936 (+ 7339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:35.758942 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:35.758947 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:35.759566 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:35.760772 (+ 1206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 00:18:35.760777 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 00:18:35.760781 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 00:18:35.760941 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 00:18:35.764755 (+ 3814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 00:18:35.764760 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 00:18:35.764921 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 00:18:35.765059 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 00:18:35.765389 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325}
I20260811 00:18:35.766959 15269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.739062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40960 (local address 127.7.94.60:42459)
0811 00:18:35.739255 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:35.739262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:35.739277 (+ 15us) server_negotiation.cc:407] Connection header received
0811 00:18:35.739326 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:35.739331 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:35.739395 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:35.739483 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:35.740583 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.741762 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.743691 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.744404 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.744532 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.751894 (+ 7362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:35.751904 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:35.751911 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:35.751954 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:35.758500 (+ 6546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.759746 (+ 1246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.759752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.759755 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.759923 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.761114 (+ 1191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.761118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.761122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.761520 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:35.766160 (+ 4640us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:35.766300 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 00:18:35.766596 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 00:18:35.771026 15273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:35.725632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40954 (local address 127.7.94.60:42459)
0811 00:18:35.725835 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:35.725842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:35.735859 (+ 10017us) server_negotiation.cc:407] Connection header received
0811 00:18:35.736309 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:35.736315 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:35.736453 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:35.736542 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:35.743487 (+ 6945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.744566 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.747855 (+ 3289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:35.748596 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:35.748705 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:35.752444 (+ 3739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 00:18:35.752452 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 00:18:35.752456 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 00:18:35.752506 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 00:18:35.756575 (+ 4069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.768444 (+ 11869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.768450 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.768454 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.768637 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 00:18:35.769156 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 00:18:35.769164 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 00:18:35.769168 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 00:18:35.769594 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 00:18:35.769901 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:35.770476 (+ 575us) server_negotiation.cc:299] Negotiation successful
0811 00:18:35.770707 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 00:18:35.776072 15131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf80e0c67a884972ad25ce38fa67312f: Bootstrap starting.
I20260811 00:18:35.780699 15214 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c [term 1 FOLLOWER]: Refusing update from remote peer a6a303d651a04a6da3b9ff15668a403f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:18:35.781812 15307 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:18:35.785691 15131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf80e0c67a884972ad25ce38fa67312f: Neither blocks nor log segments found. Creating new log.
I20260811 00:18:35.790624 15131 log.cc:824] T 00000000000000000000000000000000 P bf80e0c67a884972ad25ce38fa67312f: Log is configured to *not* fsync() on all Append() calls
I20260811 00:18:35.793022 15300 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6a303d651a04a6da3b9ff15668a403f. Latest consensus state: current_term: 1 leader_uuid: "a6a303d651a04a6da3b9ff15668a403f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a303d651a04a6da3b9ff15668a403f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } } peers { permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } } peers { permanent_uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } } }
I20260811 00:18:35.793413 15300 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:18:35.797724 15131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf80e0c67a884972ad25ce38fa67312f: No bootstrap required, opened a new log
I20260811 00:18:35.802467 15317 catalog_manager.cc:1347] Generated new cluster ID: d425389c7e6b4ca3931d41a691166a26
I20260811 00:18:35.802590 15317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 00:18:35.803056 15307 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a6a303d651a04a6da3b9ff15668a403f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a303d651a04a6da3b9ff15668a403f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } } peers { permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } } peers { permanent_uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } } }
I20260811 00:18:35.803362 15307 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:35.804970 15300 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a6a303d651a04a6da3b9ff15668a403f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a303d651a04a6da3b9ff15668a403f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } } peers { permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } } peers { permanent_uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } } }
I20260811 00:18:35.805325 15300 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:18:35.817554 15131 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf80e0c67a884972ad25ce38fa67312f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a303d651a04a6da3b9ff15668a403f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } } peers { permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } } peers { permanent_uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } }
I20260811 00:18:35.818183 15131 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf80e0c67a884972ad25ce38fa67312f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf80e0c67a884972ad25ce38fa67312f, State: Initialized, Role: FOLLOWER
I20260811 00:18:35.819172 15131 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf80e0c67a884972ad25ce38fa67312f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a303d651a04a6da3b9ff15668a403f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } } peers { permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } } peers { permanent_uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } }
I20260811 00:18:35.813759 15141 master_runner.cc:186] Error getting master registration for 127.7.94.61:44375: OK, instance_id { permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" instance_seqno: 1786407515008675 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 00:18:35.833470 15141 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 00:18:35.847904 15346 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf80e0c67a884972ad25ce38fa67312f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a303d651a04a6da3b9ff15668a403f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } } peers { permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } } peers { permanent_uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } } }
I20260811 00:18:35.848457 15346 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf80e0c67a884972ad25ce38fa67312f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:18:35.848583 15086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a6a303d651a04a6da3b9ff15668a403f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bf80e0c67a884972ad25ce38fa67312f"
I20260811 00:18:35.849109 15086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf80e0c67a884972ad25ce38fa67312f [term 0 FOLLOWER]: Advancing to term 1
I20260811 00:18:35.850337 15340 catalog_manager.cc:1259] Loaded cluster ID: d425389c7e6b4ca3931d41a691166a26
I20260811 00:18:35.850448 15340 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c: loading cluster ID for follower catalog manager: success
W20260811 00:18:35.851765 15340 catalog_manager.cc:1576] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260811 00:18:35.851897 15340 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260811 00:18:35.852917 15086 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bf80e0c67a884972ad25ce38fa67312f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6a303d651a04a6da3b9ff15668a403f in term 1.
I20260811 00:18:35.853395 15317 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 00:18:35.853902 15085 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bf80e0c67a884972ad25ce38fa67312f [term 1 FOLLOWER]: Refusing update from remote peer a6a303d651a04a6da3b9ff15668a403f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 00:18:35.854736 15131 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf80e0c67a884972ad25ce38fa67312f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 00:18:35.854897 15307 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 00:18:35.855261 15317 catalog_manager.cc:1504] Loading token signing keys...
I20260811 00:18:35.889896 15317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f: Generated new TSK 0
I20260811 00:18:35.891368 15317 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 00:18:35.930501 15346 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf80e0c67a884972ad25ce38fa67312f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6a303d651a04a6da3b9ff15668a403f. Latest consensus state: current_term: 1 leader_uuid: "a6a303d651a04a6da3b9ff15668a403f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a303d651a04a6da3b9ff15668a403f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } } peers { permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } } peers { permanent_uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } } }
I20260811 00:18:35.930573 15277 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a6a303d651a04a6da3b9ff15668a403f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a303d651a04a6da3b9ff15668a403f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } } peers { permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } } peers { permanent_uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } } }
I20260811 00:18:35.930912 15346 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf80e0c67a884972ad25ce38fa67312f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:18:35.930930 15277 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6a303d651a04a6da3b9ff15668a403f [sys.catalog]: This master's current role is: LEADER
I20260811 00:18:35.930531 15352 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf80e0c67a884972ad25ce38fa67312f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a6a303d651a04a6da3b9ff15668a403f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6a303d651a04a6da3b9ff15668a403f" member_type: VOTER last_known_addr { host: "127.7.94.62" port: 43477 } } peers { permanent_uuid: "bf80e0c67a884972ad25ce38fa67312f" member_type: VOTER last_known_addr { host: "127.7.94.61" port: 44375 } } peers { permanent_uuid: "e3c44dd0c8b64b10b9a2d74a84294e4c" member_type: VOTER last_known_addr { host: "127.7.94.60" port: 42459 } } }
I20260811 00:18:35.931058 15352 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf80e0c67a884972ad25ce38fa67312f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 00:18:35.973625 15369 catalog_manager.cc:1259] Loaded cluster ID: d425389c7e6b4ca3931d41a691166a26
I20260811 00:18:35.973738 15369 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bf80e0c67a884972ad25ce38fa67312f: loading cluster ID for follower catalog manager: success
I20260811 00:18:35.978142 15369 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bf80e0c67a884972ad25ce38fa67312f: acquiring CA information for follower catalog manager: success
I20260811 00:18:35.980904 15369 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bf80e0c67a884972ad25ce38fa67312f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 00:18:36.019747 7544 mini_kdc.cc:348] Time spent kdestroy: real 0.112s user 0.000s sys 0.032s
W20260811 00:18:36.083040 15270 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:36.071344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42708 (local address 127.7.94.62:43477)
0811 00:18:36.071575 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.071583 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.081099 (+ 9516us) server_negotiation.cc:407] Connection header received
0811 00:18:36.081408 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.081415 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.081511 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.081588 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:36.082794 (+ 1206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42708: BlockingRecv error: recv got EOF from 127.0.0.1:42708 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 00:18:36.085515 15138 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:36.081363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52850 (local address 127.7.94.61:44375)
0811 00:18:36.081584 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.081592 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.083853 (+ 2261us) server_negotiation.cc:407] Connection header received
0811 00:18:36.084105 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.084111 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.084189 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.084260 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:36.085278 (+ 1018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52850: BlockingRecv error: recv got EOF from 127.0.0.1:52850 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 00:18:36.096469 15273 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:36.093429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40970 (local address 127.7.94.60:42459)
0811 00:18:36.093703 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.093709 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.094500 (+ 791us) server_negotiation.cc:407] Connection header received
0811 00:18:36.094783 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.094789 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.094878 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.094956 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:36.096197 (+ 1241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40970: BlockingRecv error: recv got EOF from 127.0.0.1:40970 (error 108)
Metrics: {"server-negotiator.queue_time_us":141}
I20260811 00:18:36.143247 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:36.125813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52854 (local address 127.7.94.61:44375)
0811 00:18:36.126026 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.126033 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.133637 (+ 7604us) server_negotiation.cc:407] Connection header received
0811 00:18:36.134293 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.134301 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.134599 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.134753 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:36.135575 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:36.136919 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:36.139778 (+ 2859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:36.140548 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:36.140661 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:36.141539 (+ 878us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:36.142312 (+ 773us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:36.142448 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:36.142775 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 00:18:36.142882 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 00:18:36.200871 15273 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:36.187378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40982 (local address 127.7.94.60:42459)
0811 00:18:36.187631 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.187639 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.198404 (+ 10765us) server_negotiation.cc:407] Connection header received
0811 00:18:36.199060 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.199067 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.199211 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.199307 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:36.200603 (+ 1296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40982: BlockingRecv error: recv got EOF from 127.0.0.1:40982 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 00:18:36.252440 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:36.236720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52860 (local address 127.7.94.61:44375)
0811 00:18:36.236944 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.236951 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.244389 (+ 7438us) server_negotiation.cc:407] Connection header received
0811 00:18:36.244936 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.244944 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.245055 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.245132 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:36.245922 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:36.247266 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:36.249798 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:36.250603 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:36.250734 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:36.251437 (+ 703us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:36.251633 (+ 196us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:36.251738 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:36.251998 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 00:18:36.252104 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 00:18:36.297675 15273 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:36.287553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40984 (local address 127.7.94.60:42459)
0811 00:18:36.287795 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.287803 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.295315 (+ 7512us) server_negotiation.cc:407] Connection header received
0811 00:18:36.295893 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.295900 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.296006 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.296102 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:36.297354 (+ 1252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40984: BlockingRecv error: recv got EOF from 127.0.0.1:40984 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 00:18:36.363442 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:36.347997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52876 (local address 127.7.94.61:44375)
0811 00:18:36.348193 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.348216 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.356463 (+ 8247us) server_negotiation.cc:407] Connection header received
0811 00:18:36.356725 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.356732 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.356835 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.356923 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:36.357664 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:36.358869 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:36.361029 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:36.361733 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:36.361834 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:36.362509 (+ 675us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:36.362656 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:36.362726 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:36.362963 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 00:18:36.363050 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 00:18:36.424428 15273 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:36.411400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40994 (local address 127.7.94.60:42459)
0811 00:18:36.411609 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.411616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.422363 (+ 10747us) server_negotiation.cc:407] Connection header received
0811 00:18:36.422909 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.422917 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.423022 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.423107 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:36.424168 (+ 1061us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40994: BlockingRecv error: recv got EOF from 127.0.0.1:40994 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 00:18:36.459945 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:36.451414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52880 (local address 127.7.94.61:44375)
0811 00:18:36.451776 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.451784 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.452921 (+ 1137us) server_negotiation.cc:407] Connection header received
0811 00:18:36.453158 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.453165 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.453284 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.453375 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:36.454254 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:36.455497 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:36.457529 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:36.458328 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:36.458479 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:36.459013 (+ 534us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:36.459184 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:36.459277 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:36.459527 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 00:18:36.459630 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
W20260811 00:18:36.470475 15273 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:36.468187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40996 (local address 127.7.94.60:42459)
0811 00:18:36.468422 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.468428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.468730 (+ 302us) server_negotiation.cc:407] Connection header received
0811 00:18:36.469069 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.469075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.469180 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.469302 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:36.470282 (+ 980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40996: BlockingRecv error: recv got EOF from 127.0.0.1:40996 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
I20260811 00:18:36.506595 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:36.500787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52890 (local address 127.7.94.61:44375)
0811 00:18:36.501039 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.501048 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.501697 (+ 649us) server_negotiation.cc:407] Connection header received
0811 00:18:36.501929 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.501935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.502022 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.502133 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:36.502596 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:36.503390 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:36.504751 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:36.505270 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:36.505417 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:36.505825 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:36.505947 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:36.506044 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:36.506181 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 00:18:36.506259 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260811 00:18:36.514541 15273 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:36.512592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41002 (local address 127.7.94.60:42459)
0811 00:18:36.512814 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.512822 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.513144 (+ 322us) server_negotiation.cc:407] Connection header received
0811 00:18:36.513307 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.513312 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.513409 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.513524 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:36.514261 (+ 737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41002: BlockingRecv error: recv got EOF from 127.0.0.1:41002 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 00:18:36.540643 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:36.534192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52892 (local address 127.7.94.61:44375)
0811 00:18:36.534419 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.534426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.534798 (+ 372us) server_negotiation.cc:407] Connection header received
0811 00:18:36.535004 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.535009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.535076 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.535180 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:36.535665 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:36.536495 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:36.538462 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:36.539125 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:36.539224 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:36.539836 (+ 612us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:36.539998 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:36.540128 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:36.540281 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 00:18:36.540353 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 00:18:36.557761 15273 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:36.555314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41010 (local address 127.7.94.60:42459)
0811 00:18:36.555570 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.555578 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.556206 (+ 628us) server_negotiation.cc:407] Connection header received
0811 00:18:36.556437 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.556440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.556516 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.556611 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:36.557539 (+ 928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41010: BlockingRecv error: recv got EOF from 127.0.0.1:41010 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 00:18:36.601154 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:36.593990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52908 (local address 127.7.94.61:44375)
0811 00:18:36.594216 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.594223 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.594870 (+ 647us) server_negotiation.cc:407] Connection header received
0811 00:18:36.595087 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.595093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.595190 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.595289 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:36.595833 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:36.597034 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:36.598955 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:36.599750 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:36.599885 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:36.600355 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:36.600498 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:36.600593 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:36.600769 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 00:18:36.600845 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 00:18:36.609102 15273 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:36.607267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41024 (local address 127.7.94.60:42459)
0811 00:18:36.607457 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.607463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.607773 (+ 310us) server_negotiation.cc:407] Connection header received
0811 00:18:36.607900 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.607905 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.608031 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.608175 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:36.608902 (+ 727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41024: BlockingRecv error: recv got EOF from 127.0.0.1:41024 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 00:18:36.681944 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:36.676505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52912 (local address 127.7.94.61:44375)
0811 00:18:36.676775 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.676782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.677014 (+ 232us) server_negotiation.cc:407] Connection header received
0811 00:18:36.677201 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.677205 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.677314 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.677388 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:36.677784 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:36.678537 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:36.680113 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:36.680569 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:36.680666 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:36.681038 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:36.681127 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:36.681205 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:36.681479 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 00:18:36.681560 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
W20260811 00:18:36.689679 15273 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:36.687869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41036 (local address 127.7.94.60:42459)
0811 00:18:36.688151 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.688158 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.688403 (+ 245us) server_negotiation.cc:407] Connection header received
0811 00:18:36.688546 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.688549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.688621 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.688704 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:36.689465 (+ 761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41036: BlockingRecv error: recv got EOF from 127.0.0.1:41036 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 00:18:36.827615 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:36.822899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52918 (local address 127.7.94.61:44375)
0811 00:18:36.823150 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.823170 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.823370 (+ 200us) server_negotiation.cc:407] Connection header received
0811 00:18:36.823533 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.823537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.823607 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.823688 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:36.824127 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:36.824817 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:36.826112 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:36.826527 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:36.826623 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:36.826960 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:36.827058 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:36.827121 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:36.827246 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 00:18:36.827306 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260811 00:18:36.835469 15273 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 00:18:36.833427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41048 (local address 127.7.94.60:42459)
0811 00:18:36.833617 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:36.833622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:36.834053 (+ 431us) server_negotiation.cc:407] Connection header received
0811 00:18:36.834266 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:36.834271 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:36.834343 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:36.834446 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 00:18:36.835252 (+ 806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41048: BlockingRecv error: recv got EOF from 127.0.0.1:41048 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 00:18:36.855430 15340 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c: acquiring CA information for follower catalog manager: success
I20260811 00:18:36.857486 15340 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e3c44dd0c8b64b10b9a2d74a84294e4c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 00:18:37.100281 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:37.095279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52926 (local address 127.7.94.61:44375)
0811 00:18:37.095537 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:37.095544 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:37.095741 (+ 197us) server_negotiation.cc:407] Connection header received
0811 00:18:37.095987 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:37.095991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:37.096058 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:37.096167 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:37.096587 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:37.097365 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:37.098677 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:37.099069 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:37.099216 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:37.099597 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:37.099709 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:37.099807 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:37.099961 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 00:18:37.100036 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 00:18:37.110939 15273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:37.105500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41052 (local address 127.7.94.60:42459)
0811 00:18:37.105724 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:37.105731 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:37.105974 (+ 243us) server_negotiation.cc:407] Connection header received
0811 00:18:37.106110 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:37.106114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:37.106275 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:37.106373 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:37.106695 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:37.107387 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:37.108586 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:37.108974 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:37.109063 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:37.109572 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:37.110314 (+ 742us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:37.110488 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:37.110534 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 00:18:37.110641 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 00:18:37.124790 15556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:37.116602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42716 (local address 127.7.94.62:43477)
0811 00:18:37.117099 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:37.117106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:37.117230 (+ 124us) server_negotiation.cc:407] Connection header received
0811 00:18:37.117520 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:37.117531 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:37.117699 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:37.117876 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:37.118647 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:37.119829 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:37.121587 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:37.122011 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:37.122146 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:37.123354 (+ 1208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:37.123815 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:37.123929 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:37.124385 (+ 456us) server_negotiation.cc:299] Negotiation successful
0811 00:18:37.124471 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":261,"threads_started":1}
I20260811 00:18:37.124782 15273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:37.117922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41068 (local address 127.7.94.60:42459)
0811 00:18:37.118097 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:37.118103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:37.118670 (+ 567us) server_negotiation.cc:407] Connection header received
0811 00:18:37.118867 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:37.118871 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:37.118927 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:37.119016 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:37.119485 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:37.120340 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:37.122606 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:37.123094 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:37.123218 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:37.123783 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:37.123905 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:37.123987 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:37.124430 (+ 443us) server_negotiation.cc:299] Negotiation successful
0811 00:18:37.124497 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 00:18:37.127173 15138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 00:18:37.117308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52928 (local address 127.7.94.61:44375)
0811 00:18:37.117481 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 00:18:37.117487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 00:18:37.117879 (+ 392us) server_negotiation.cc:407] Connection header received
0811 00:18:37.118243 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 00:18:37.118252 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 00:18:37.118347 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 00:18:37.118417 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 00:18:37.119158 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:37.120095 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:37.122552 (+ 2457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 00:18:37.122981 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 00:18:37.123134 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 00:18:37.124043 (+ 909us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 00:18:37.124196 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 00:18:37.124308 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 00:18:37.126718 (+ 2410us) server_negotiation.cc:299] Negotiation successful
0811 00:18:37.126824 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 00:18:37.130689 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 14889
I20260811 00:18:37.131569 14997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:18:37.240938 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 14889
W20260811 00:18:37.269553 15167 connection.cc:535] server connection from 127.0.0.1:40934 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 00:18:37.270020 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 15016
I20260811 00:18:37.271116 15123 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:18:37.378720 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 15016
I20260811 00:18:37.408061 7544 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task1KEo_S/build/asan/bin/kudu pid 15141
I20260811 00:18:37.409298 15255 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 00:18:37.517462 7544 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1KEo_S/build/asan/bin/kudu with pid 15141
2026-08-11T00:18:37Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5225 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10005 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (383535 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST